-- Logs begin at Wed 2022-06-29 10:32:34 UTC, end at Wed 2022-06-29 11:52:27 UTC. -- Jun 29 11:05:12.870183 dv-cinder-847335-2-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jun 29 11:05:12.877469 dv-cinder-847335-2-iscsi devstack@keystone.service[80153]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jun 29 11:05:12.879496 dv-cinder-847335-2-iscsi devstack@keystone.service[80153]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed Jun 29 11:05:12 2022] *** Jun 29 11:05:12.879496 dv-cinder-847335-2-iscsi devstack@keystone.service[80153]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jun 29 11:05:12.879496 dv-cinder-847335-2-iscsi devstack@keystone.service[80153]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jun 29 11:05:12.879496 dv-cinder-847335-2-iscsi devstack@keystone.service[80153]: nodename: dv-cinder-847335-2-iscsi Jun 29 11:05:12.879496 dv-cinder-847335-2-iscsi devstack@keystone.service[80153]: machine: x86_64 Jun 29 11:05:12.879496 dv-cinder-847335-2-iscsi devstack@keystone.service[80153]: clock source: unix Jun 29 11:05:12.879496 dv-cinder-847335-2-iscsi devstack@keystone.service[80153]: pcre jit disabled Jun 29 11:05:12.879911 dv-cinder-847335-2-iscsi devstack@keystone.service[80153]: detected number of CPU cores: 4 Jun 29 11:05:12.879911 dv-cinder-847335-2-iscsi devstack@keystone.service[80153]: current working directory: / Jun 29 11:05:12.879911 dv-cinder-847335-2-iscsi devstack@keystone.service[80153]: detected binary path: /usr/bin/uwsgi-core Jun 29 11:05:12.879911 dv-cinder-847335-2-iscsi devstack@keystone.service[80153]: your processes number limit is 47680 Jun 29 11:05:12.879911 dv-cinder-847335-2-iscsi devstack@keystone.service[80153]: your memory page size is 4096 bytes Jun 29 11:05:12.879911 dv-cinder-847335-2-iscsi devstack@keystone.service[80153]: detected max file descriptor number: 2048 Jun 29 11:05:12.879911 dv-cinder-847335-2-iscsi devstack@keystone.service[80153]: lock engine: pthread robust mutexes Jun 29 11:05:12.879911 dv-cinder-847335-2-iscsi devstack@keystone.service[80153]: thunder lock: enabled Jun 29 11:05:12.882338 dv-cinder-847335-2-iscsi devstack@keystone.service[80153]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jun 29 11:05:12.882338 dv-cinder-847335-2-iscsi devstack@keystone.service[80153]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] Jun 29 11:05:12.924253 dv-cinder-847335-2-iscsi devstack@keystone.service[80153]: Python main interpreter initialized at 0x55ddc8de19c0 Jun 29 11:05:12.924253 dv-cinder-847335-2-iscsi devstack@keystone.service[80153]: python threads support enabled Jun 29 11:05:12.924429 dv-cinder-847335-2-iscsi devstack@keystone.service[80153]: your server socket listen backlog is limited to 100 connections Jun 29 11:05:12.924429 dv-cinder-847335-2-iscsi devstack@keystone.service[80153]: your mercy for graceful operations on workers is 90 seconds Jun 29 11:05:12.924593 dv-cinder-847335-2-iscsi devstack@keystone.service[80153]: mapped 403077 bytes (393 KB) for 2 cores Jun 29 11:05:12.924716 dv-cinder-847335-2-iscsi devstack@keystone.service[80153]: *** Operational MODE: preforking *** Jun 29 11:05:12.924996 dv-cinder-847335-2-iscsi devstack@keystone.service[80153]: *** uWSGI is running in multiple interpreter mode *** Jun 29 11:05:12.924996 dv-cinder-847335-2-iscsi devstack@keystone.service[80153]: spawned uWSGI master process (pid: 80153) Jun 29 11:05:12.925186 dv-cinder-847335-2-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jun 29 11:05:12.926166 dv-cinder-847335-2-iscsi devstack@keystone.service[80153]: spawned uWSGI worker 1 (pid: 80165, cores: 1) Jun 29 11:05:12.926166 dv-cinder-847335-2-iscsi devstack@keystone.service[80153]: spawned uWSGI worker 2 (pid: 80166, cores: 1) Jun 29 11:05:12.926700 dv-cinder-847335-2-iscsi devstack@keystone.service[80153]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jun 29 11:05:18.415988 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 29 11:05:18.415988 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: from cryptography.utils import int_from_bytes Jun 29 11:05:18.416813 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 29 11:05:18.416813 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: from cryptography.utils import int_from_bytes Jun 29 11:05:18.419737 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 29 11:05:18.419737 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: from cryptography.utils import int_from_bytes Jun 29 11:05:18.420189 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 29 11:05:18.420189 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: from cryptography.utils import int_from_bytes Jun 29 11:05:18.469537 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 29 11:05:18.469735 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 29 11:05:18.469929 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] command line args: [] {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 29 11:05:18.470197 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 29 11:05:18.470377 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ================================================================================ {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 29 11:05:18.470631 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] admin_token = **** {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 11:05:18.470877 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] config_dir = [] {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 11:05:18.471064 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 11:05:18.471246 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] config_source = [] {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 11:05:18.471427 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] debug = True {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 11:05:18.471624 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 11:05:18.471885 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 29 11:05:18.471885 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 29 11:05:18.472021 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 11:05:18.472086 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] command line args: [] {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 29 11:05:18.472223 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 11:05:18.472298 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 29 11:05:18.472424 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 11:05:18.472503 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] ================================================================================ {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 29 11:05:18.472672 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 11:05:18.472782 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] admin_token = **** {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 11:05:18.472890 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] list_limit = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 11:05:18.472991 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] config_dir = [] {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 11:05:18.473097 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] log_config_append = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 11:05:18.473198 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 11:05:18.473306 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 11:05:18.473407 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] config_source = [] {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 11:05:18.473501 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] log_dir = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 11:05:18.473600 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] debug = True {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 11:05:18.473702 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] log_file = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 11:05:18.473813 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 11:05:18.473962 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 11:05:18.474044 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 11:05:18.474159 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 11:05:18.474232 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 11:05:18.474349 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 11:05:18.474434 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 11:05:18.474511 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 11:05:18.474636 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 11:05:18.474711 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 11:05:18.474824 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] list_limit = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 11:05:18.474904 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 11:05:18.474987 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] log_config_append = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 11:05:18.475107 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 11:05:18.475187 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 11:05:18.475300 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] log_dir = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 11:05:18.475371 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 11:05:18.475492 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] log_file = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 11:05:18.475572 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 11:05:18.475692 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 11:05:18.475773 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 11:05:18.475864 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 11:05:18.475972 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 11:05:18.476110 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 11:05:18.476194 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 11:05:18.476358 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 11:05:18.476438 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 11:05:18.476552 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 11:05:18.476663 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 11:05:18.476783 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 11:05:18.476925 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 11:05:18.477049 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 11:05:18.477160 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 11:05:18.477397 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 11:05:18.477519 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.44/identity {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 11:05:18.477662 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] publish_errors = False {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 11:05:18.477772 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 11:05:18.477882 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 11:05:18.478004 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 11:05:18.478113 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 11:05:18.478223 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 11:05:18.478339 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 11:05:18.478456 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 11:05:18.478567 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 11:05:18.478726 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 11:05:18.478847 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 11:05:18.478964 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 11:05:18.479083 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] standard_threads = False {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 11:05:18.479083 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 11:05:18.479297 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.44/identity {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 11:05:18.479416 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 11:05:18.479547 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] publish_errors = False {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 11:05:18.479660 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 11:05:18.479660 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] use_journal = False {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 11:05:18.479908 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 11:05:18.480023 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] use_json = False {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 11:05:18.480130 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 11:05:18.480241 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] use_stderr = False {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 11:05:18.480357 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 11:05:18.480477 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] use_syslog = False {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 11:05:18.480595 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 11:05:18.480719 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 11:05:18.480838 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 11:05:18.480950 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.481066 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] standard_threads = False {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 11:05:18.481187 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.481305 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 11:05:18.481424 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.481535 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 11:05:18.481656 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.481768 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 11:05:18.481768 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] use_journal = False {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 11:05:18.482058 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.482169 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] use_json = False {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 11:05:18.482279 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.482389 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] use_stderr = False {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 11:05:18.482506 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.482626 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] use_syslog = False {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 11:05:18.482738 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.482849 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 11:05:18.482960 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.483077 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.483190 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.483309 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.483422 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.483532 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.483642 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.483752 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.483876 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.484000 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.484105 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.484214 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.484331 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.484442 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.484564 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.484684 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.484803 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.484915 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.485032 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.485149 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.485272 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.485390 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.485502 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.485608 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.485700 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.485775 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.485846 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.485904 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.485904 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.486122 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.486233 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.486344 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.486463 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.486582 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.486693 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.486804 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.486913 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.487023 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.487209 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.487325 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.487441 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.487559 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.487676 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.487793 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.487793 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.488032 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.488142 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.488254 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.488370 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.488488 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.488608 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.488719 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.488830 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.488940 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.489050 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.489166 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.489287 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.489399 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.489515 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.489626 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.489736 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.489853 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.489973 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.490085 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.490201 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.490318 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.490429 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.490539 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.490641 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.490737 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.490829 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.490922 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.491050 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.491159 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.491261 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.491357 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.491458 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.491558 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.491681 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.491802 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.491945 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.492059 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.492161 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.492272 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.492383 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.492503 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.492982 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.493102 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.493718 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.493840 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.493967 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.494081 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.494194 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.494319 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.494433 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.494546 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.494658 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.494769 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.494880 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.495001 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.495120 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.495239 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.495342 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.495453 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.495573 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.495686 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.495797 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.495932 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.496036 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.501122 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.501248 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.501336 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.501414 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.501491 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.501568 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.501639 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.501709 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.501786 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.501859 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.501974 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.502048 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.502118 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.502197 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.502269 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.502338 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.502407 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.502488 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.502560 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.502630 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.502707 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.502778 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.502847 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.502922 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.502992 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.503061 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.503201 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.503280 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.503356 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.503427 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.503497 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.503566 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.503643 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.503714 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.503783 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.503867 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.503985 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.504059 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.504146 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.504229 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.504301 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.504370 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.504439 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.504508 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.504584 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.504662 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.504733 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.504803 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.504872 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.504948 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.505024 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.505103 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.505205 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.505281 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.505359 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.505430 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.505500 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.505578 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.505655 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.505726 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.505795 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.505864 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.505934 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.506011 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.506088 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.506159 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.506229 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.506299 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.506369 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.506446 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.506522 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.506591 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.506661 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.506729 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.506798 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.506875 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.506946 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.507016 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.507086 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.507156 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.507225 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.507311 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.507383 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.507453 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] auth.external = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.507522 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.507590 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.507659 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.507737 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.507809 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.508015 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.508087 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.508190 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] auth.password = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.508325 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.508438 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] auth.token = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.508536 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.508631 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.508761 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.509442 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.509551 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.509648 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.509744 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] auth.external = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.509861 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.509980 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.510057 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.510128 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.510209 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.510282 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.510360 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.510433 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] auth.password = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.510502 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] credential.caching = True {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.510572 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] auth.token = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.510642 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.510713 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.510790 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.510863 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.510933 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.511003 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.511080 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.511159 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.511231 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.511302 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.511372 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.511442 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.511512 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.511590 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.511668 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.511739 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] credential.caching = True {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.511809 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.511939 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.512015 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.512099 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.512184 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.512257 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.512328 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.512399 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.512469 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.512550 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.512623 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.512693 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.512770 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.512843 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.512913 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] federation.caching = True {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.513058 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.513133 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.513204 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.513275 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.513351 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.513423 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.513504 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.513577 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.513648 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.513718 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.513789 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.513892 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.514075 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.514156 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.514229 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.514300 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.514415 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] federation.caching = True {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.514498 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.514655 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.514736 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.514904 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.514984 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.515154 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.515234 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] identity.caching = True {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.515387 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.515467 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.515636 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.515715 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.515894 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.515987 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.516136 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.516216 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.516380 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.516457 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.516609 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.516685 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.516847 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.516987 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.517096 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.517217 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.517335 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] identity.caching = True {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.517448 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.517570 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.517685 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.517805 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.517919 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.518033 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.518146 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.518268 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.518382 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.518495 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.518608 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.518728 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.518843 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.518963 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.519077 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.519198 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.519343 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.519522 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.519644 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.519760 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.519887 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.520002 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.520115 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.520241 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.520365 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.520481 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.520594 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.520707 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.520820 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.520932 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.521053 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.521168 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.521282 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.521395 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.521508 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.521621 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.521742 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.521858 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.521971 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.522084 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.522196 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.522308 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.522429 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.522543 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.522656 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.522769 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.522883 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.522996 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.523117 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.523231 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.523344 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.523457 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.523569 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.523682 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.523806 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.523966 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.524080 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.524193 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.524306 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.524438 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.524567 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.524673 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.524673 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.524885 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.524999 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.525111 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.525231 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.525346 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.525476 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.525589 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.525702 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.525814 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.525935 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.526051 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.526163 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.526275 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.526388 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.526500 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.526622 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.526736 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.526848 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.526990 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.527108 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.user = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.527221 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.527342 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.527461 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.527575 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.527687 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.527800 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.527929 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.528063 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.528181 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.528294 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.528407 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.528520 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.528632 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.528753 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.528890 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.529010 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.529010 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.529224 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.529336 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.529458 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.529573 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.529685 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] ldap.user = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.529797 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.529909 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.530021 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.530142 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.530257 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.530370 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.530482 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.530594 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.530706 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.530826 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.530946 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.531059 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.531171 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.531284 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.531411 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.531579 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.531714 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.531868 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.532024 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.532144 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.532283 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.532389 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.532502 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.532615 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.532727 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.532839 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.532960 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.533075 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.533188 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.533301 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.533414 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.533526 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.533647 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.533761 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.533874 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.533987 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.534100 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.534212 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.534333 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.534449 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.534563 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.534675 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.534788 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.534901 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.535086 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.535204 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.535317 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.535431 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.535543 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.535656 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.535776 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] resource.caching = True {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.535954 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.536068 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.536181 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.536294 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.536407 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.536533 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.536649 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.536762 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.536875 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.536987 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.537100 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.537221 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.537335 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.537453 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.537567 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.537680 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.537793 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.537914 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.538028 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.538140 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] role.caching = True {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.538253 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.538365 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] role.driver = sql {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.538478 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] resource.caching = True {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.538600 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.538714 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.538827 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.538940 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.539052 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.539166 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.539287 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.539401 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.539513 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.539628 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.539741 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.539876 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.540017 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.540157 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.540271 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.540384 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.540496 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.540496 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.540718 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.540833 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.540946 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] role.caching = True {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.541059 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.541173 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] role.driver = sql {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.541286 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.541407 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.541529 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.541643 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.541643 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.541856 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.541970 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.542091 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.542206 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.542319 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.542432 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.542545 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.542666 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.542782 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.542895 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.543008 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.543122 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.543234 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.543355 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.543470 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.543589 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.543703 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.543817 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.543964 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.544090 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.544206 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.544319 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.544432 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.544432 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.544644 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.544766 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.544882 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.544995 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.545107 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.545220 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.545333 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.545454 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.545568 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.545681 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.545793 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.545907 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.546026 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.546149 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] token.caching = True {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.546262 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.546376 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.546488 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.546601 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.546714 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.546835 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.546950 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.547063 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.547176 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.547289 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.547404 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.547527 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.547641 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.547755 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.547881 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.547997 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.548110 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.548235 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.548350 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.548464 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.548576 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.548689 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.548801 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.548923 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.549038 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] token.caching = True {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.549151 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.549264 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.549376 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.549497 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.549612 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.549724 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.549838 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.549971 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.550085 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.550206 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.550321 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] database.connection = **** {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.550443 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.550565 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.550680 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.550793 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.550906 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.551019 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.551141 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.551255 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.551367 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.551480 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.551592 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.551705 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.551844 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.551961 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.552074 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.552188 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.552301 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.552413 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.552538 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.552653 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.552766 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.552885 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.552999 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.553111 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.553238 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] database.connection = **** {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.553362 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.553550 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.553674 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.553788 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.553902 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.554018 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.554140 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.554259 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.554374 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.554486 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.554599 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.554599 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 29 11:05:18.554836 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.554836 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.554836 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.555152 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.555263 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.555424 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.556217 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.556457 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.556696 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.556943 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.557175 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.557420 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.557658 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 11:05:18.557831 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80165) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 29 11:05:18.585572 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 11:05:18.585803 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 11:05:18.586041 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 11:05:18.587665 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80165) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 11:05:18.587923 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80165) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 11:05:18.588156 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80165) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 11:05:18.594513 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 11:05:18.596511 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80165) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 11:05:18.623289 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 11:05:18.623519 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 11:05:18.625400 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80165) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 11:05:18.625642 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80165) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 11:05:18.764934 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80165) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 11:05:18.765139 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 11:05:18.814605 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80165) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 11:05:18.814833 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80165) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 11:05:18.818269 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 11:05:18.818509 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 11:05:18.879535 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80165) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 11:05:18.879766 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80165) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 11:05:18.879868 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 11:05:18.880020 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80165) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 11:05:18.880102 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 11:05:18.880244 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80165) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 11:05:18.880344 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 11:05:18.880465 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80165) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 11:05:18.880548 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 11:05:18.880682 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80165) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 11:05:18.880763 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 11:05:18.880930 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 11:05:18.906055 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 11:05:18.906275 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80165) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 11:05:18.906366 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 11:05:18.906506 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 11:05:18.906587 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80165) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 11:05:18.906732 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 11:05:18.906815 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80165) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 11:05:18.906953 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 11:05:18.907034 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80165) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 11:05:18.907169 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80165) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 11:05:18.907258 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 11:05:18.907385 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 11:05:18.907465 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80165) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 11:05:18.907595 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80165) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 11:05:18.918627 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 11:05:18.918813 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80165) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 11:05:18.931734 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80165) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 11:05:18.931961 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 11:05:18.932052 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80165) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 11:05:18.932226 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80165) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 11:05:18.932393 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 11:05:18.932475 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80165) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 11:05:18.932616 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 11:05:18.932698 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80165) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 11:05:18.932841 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80165) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 11:05:18.932958 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 11:05:18.933041 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80165) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 11:05:18.933186 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 11:05:18.933403 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 11:05:18.933618 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 11:05:18.950871 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 11:05:18.951083 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 29 11:05:18.951621 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 11:05:18.951703 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 11:05:18.951858 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 29 11:05:18.951942 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 29 11:05:18.952132 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 11:05:18.952214 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 11:05:18.952394 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 11:05:18.952474 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 29 11:05:18.952653 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 11:05:18.952734 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 11:05:18.952911 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 11:05:18.952993 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 11:05:18.953210 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 11:05:18.953467 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 11:05:18.963730 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 11:05:18.964037 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 11:05:18.964306 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 11:05:18.964503 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 11:05:18.964583 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 11:05:18.964779 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 11:05:18.965044 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 11:05:18.965323 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 11:05:18.972766 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80165) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 11:05:18.973634 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80166) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 11:05:18.976059 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80165) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 11:05:18.976344 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 11:05:18.976627 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 11:05:18.976911 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 11:05:18.976991 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80166) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 11:05:18.977189 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 11:05:18.977268 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 11:05:18.977449 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 11:05:18.977530 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 11:05:18.977744 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 11:05:18.977825 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 11:05:18.978018 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 11:05:18.978108 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 11:05:18.978284 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 11:05:18.978363 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 11:05:18.978554 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 11:05:18.978633 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 11:05:18.978819 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 11:05:18.978906 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 11:05:18.979085 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 11:05:18.979357 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 11:05:18.979623 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 11:05:19.005821 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 11:05:19.006388 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 11:05:19.007425 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80165) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 11:05:19.007705 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 11:05:19.008018 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80166) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 11:05:19.008297 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 11:05:19.012987 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80165) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 11:05:19.013267 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 11:05:19.013550 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80166) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 11:05:19.013631 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 11:05:19.013829 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 11:05:19.014222 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 11:05:19.020962 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80165) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 11:05:19.021199 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 11:05:19.021468 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80166) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 11:05:19.021749 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 11:05:19.026215 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80165) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 11:05:19.026502 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 11:05:19.026773 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80166) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 11:05:19.026855 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 11:05:19.027056 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 11:05:19.027136 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 11:05:19.027332 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 11:05:19.027419 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 11:05:19.027607 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 11:05:19.027688 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 11:05:19.027903 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 11:05:19.027986 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 11:05:19.028175 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 11:05:19.028258 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 11:05:19.028442 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 11:05:19.028711 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 11:05:19.046826 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 11:05:19.047099 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 11:05:19.047182 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 11:05:19.047398 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 11:05:19.051688 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80165) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 11:05:19.052002 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80166) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 11:05:19.055319 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 11:05:19.055610 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 11:05:19.056242 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 11:05:19.056442 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 11:05:19.061249 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80165) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 11:05:19.061530 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80166) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 11:05:19.064875 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80165) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 11:05:19.065207 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80166) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 11:05:19.068549 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 11:05:19.068850 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 11:05:19.069135 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 11:05:19.069423 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 11:05:19.069718 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 11:05:19.069996 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 11:05:19.070077 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 11:05:19.070385 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 11:05:19.070674 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 11:05:19.073430 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 11:05:19.073748 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 11:05:19.074035 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 11:05:19.089725 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80165) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 11:05:19.090014 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 11:05:19.090284 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 11:05:19.090547 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 11:05:19.095474 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80166) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 11:05:19.095764 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 11:05:19.096061 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 11:05:19.096333 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 11:05:19.098433 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 11:05:19.100149 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 11:05:19.100427 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 11:05:19.103686 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80165) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 11:05:19.103998 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 11:05:19.104250 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 11:05:19.104335 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 11:05:19.104552 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 11:05:19.104836 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 11:05:19.105948 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 11:05:19.106229 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 11:05:19.109508 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80166) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 11:05:19.109793 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 11:05:19.110076 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 11:05:19.110356 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 11:05:19.110635 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 11:05:19.117976 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80165) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 11:05:19.118774 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 11:05:19.119054 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 11:05:19.119335 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 11:05:19.119625 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 11:05:19.119925 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 11:05:19.120211 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 11:05:19.124380 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80166) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 11:05:19.124661 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 11:05:19.124941 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 11:05:19.125221 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 11:05:19.125514 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 11:05:19.127193 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 11:05:19.127496 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 11:05:19.139904 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80165) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 11:05:19.143149 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80165) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 11:05:19.145827 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80166) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 11:05:19.146462 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 11:05:19.148085 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 11:05:19.149284 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80166) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 11:05:19.149864 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80165) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 11:05:19.150145 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 11:05:19.150426 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 11:05:19.152639 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 11:05:19.154383 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 11:05:19.156032 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80166) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 11:05:19.156315 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 11:05:19.156595 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 11:05:19.157753 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 11:05:19.159339 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80165) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 11:05:19.162568 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 11:05:19.162866 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 11:05:19.163153 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 11:05:19.163431 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 11:05:19.163947 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 11:05:19.165567 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80166) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 11:05:19.168798 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 11:05:19.169096 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 11:05:19.169491 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 11:05:19.169773 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 11:05:19.172640 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80165) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 11:05:19.172932 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 11:05:19.173214 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 11:05:19.178832 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80166) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 11:05:19.179121 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 11:05:19.179401 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 11:05:19.181592 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80165) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 11:05:19.181872 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 11:05:19.182154 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 11:05:19.182477 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 11:05:19.182697 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 11:05:19.182966 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 11:05:19.183244 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 11:05:19.183515 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 11:05:19.183920 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 11:05:19.184199 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 11:05:19.184479 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 11:05:19.184746 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 11:05:19.185013 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 11:05:19.185286 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80165) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 11:05:19.187747 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80166) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 11:05:19.188054 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 11:05:19.188335 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 11:05:19.188610 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 11:05:19.188882 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 11:05:19.189262 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 11:05:19.189545 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 11:05:19.189815 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 11:05:19.190110 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 11:05:19.190382 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 11:05:19.190654 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 11:05:19.190917 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 11:05:19.191181 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 11:05:19.191448 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 11:05:19.223466 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x55ddc8de19c0 pid: 80165 (default app) Jun 29 11:05:19.226883 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b8fd35-cc25-49b1-914b-99c92dd9ddde None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:19.227179 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b8fd35-cc25-49b1-914b-99c92dd9ddde None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:19.227466 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b8fd35-cc25-49b1-914b-99c92dd9ddde None None] PATH_INFO: `/v3` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:19.229653 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x55ddc8de19c0 pid: 80166 (default app) Jun 29 11:05:19.231671 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1/1] 10.222.0.44 () {50 vars in 791 bytes} [Wed Jun 29 11:05:19 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:05:27.721103 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc338139-4078-46be-a4d2-7eb6594e799c None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:27.721635 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc338139-4078-46be-a4d2-7eb6594e799c None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:27.721635 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc338139-4078-46be-a4d2-7eb6594e799c None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:27.725979 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1/2] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:05:27 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:05:27.731609 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1c6baa-6312-4e7b-a38b-ce28fa0fd019 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:27.732027 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1c6baa-6312-4e7b-a38b-ce28fa0fd019 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:27.732177 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1c6baa-6312-4e7b-a38b-ce28fa0fd019 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:27.800909 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: INFO dbcounter [None req-1b1c6baa-6312-4e7b-a38b-ce28fa0fd019 None None] Registered counter for database keystone Jun 29 11:05:27.822102 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG oslo_db.sqlalchemy.engines [None req-1b1c6baa-6312-4e7b-a38b-ce28fa0fd019 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80165) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 29 11:05:27.828496 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writer thread running {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 29 11:05:28.003038 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG passlib.handlers.bcrypt [None req-1b1c6baa-6312-4e7b-a38b-ce28fa0fd019 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80165) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 29 11:05:28.003387 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG passlib.handlers.bcrypt [None req-1b1c6baa-6312-4e7b-a38b-ce28fa0fd019 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80165) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 29 11:05:28.057200 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-1b1c6baa-6312-4e7b-a38b-ce28fa0fd019 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:05:28.058781 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-1b1c6baa-6312-4e7b-a38b-ce28fa0fd019 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:05:28.178882 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 2/3] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:05:27 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 448 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Jun 29 11:05:28.407932 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc338139-4078-46be-a4d2-7eb6594e799c None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:05:28.467700 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: INFO dbcounter [None req-fc338139-4078-46be-a4d2-7eb6594e799c None None] Registered counter for database keystone Jun 29 11:05:28.487630 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG oslo_db.sqlalchemy.engines [None req-fc338139-4078-46be-a4d2-7eb6594e799c None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80166) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 29 11:05:28.494479 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writer thread running {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 29 11:05:28.612495 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc338139-4078-46be-a4d2-7eb6594e799c None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:05:28.616763 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b0f574-9f0b-4523-83c0-0dbbc9ca06bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:05:28.617411 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b0f574-9f0b-4523-83c0-0dbbc9ca06bb admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:28.617713 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b0f574-9f0b-4523-83c0-0dbbc9ca06bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:28.618512 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b0f574-9f0b-4523-83c0-0dbbc9ca06bb admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:28.626677 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b0f574-9f0b-4523-83c0-0dbbc9ca06bb admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:05:28.678322 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: WARNING keystone.common.rbac_enforcer.enforcer [None req-e0b0f574-9f0b-4523-83c0-0dbbc9ca06bb admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 29 11:05:28.678603 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b0f574-9f0b-4523-83c0-0dbbc9ca06bb admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:05:28.689039 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 2/4] 10.222.0.44 () {58 vars in 1324 bytes} [Wed Jun 29 11:05:28 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 29 11:05:28.695041 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1c6baa-6312-4e7b-a38b-ce28fa0fd019 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:05:28.710044 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1c6baa-6312-4e7b-a38b-ce28fa0fd019 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:05:28.714999 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e3c87d-00a7-474b-906e-9fcebe0576fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:05:28.716199 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e3c87d-00a7-474b-906e-9fcebe0576fa admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:28.716488 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e3c87d-00a7-474b-906e-9fcebe0576fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:28.716764 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e3c87d-00a7-474b-906e-9fcebe0576fa admin admin] PATH_INFO: `/v3/projects/37897f6664b94d219c11b87aa269041a` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:28.720838 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e3c87d-00a7-474b-906e-9fcebe0576fa admin admin] RBAC: Authorizing `identity:get_project(project_id=37897f6664b94d219c11b87aa269041a)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:05:28.771351 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: WARNING keystone.common.rbac_enforcer.enforcer [None req-f7e3c87d-00a7-474b-906e-9fcebe0576fa admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 29 11:05:28.771927 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e3c87d-00a7-474b-906e-9fcebe0576fa admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:05:28.776267 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 3/5] 10.222.0.44 () {56 vars in 1207 bytes} [Wed Jun 29 11:05:28 2022] GET /identity/v3/projects/37897f6664b94d219c11b87aa269041a => generated 344 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:05:30.770353 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a231bfe-d30a-4e1a-bda9-d66c13bce9fd None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:30.770900 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a231bfe-d30a-4e1a-bda9-d66c13bce9fd None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:30.770900 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a231bfe-d30a-4e1a-bda9-d66c13bce9fd None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:30.775656 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 3/6] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:05:30 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:05:30.781051 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6613534-018e-490f-9977-761ad3404853 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:30.781447 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6613534-018e-490f-9977-761ad3404853 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:30.781648 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6613534-018e-490f-9977-761ad3404853 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:30.799965 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-f6613534-018e-490f-9977-761ad3404853 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:05:30.801491 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-f6613534-018e-490f-9977-761ad3404853 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:05:30.809318 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 4/7] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:05:30 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jun 29 11:05:30.832110 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a231bfe-d30a-4e1a-bda9-d66c13bce9fd None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:05:30.844673 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a231bfe-d30a-4e1a-bda9-d66c13bce9fd None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:05:30.848822 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40182c0-c3a6-4aea-9b71-0127f358e9cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:05:30.849440 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40182c0-c3a6-4aea-9b71-0127f358e9cd admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:30.849725 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40182c0-c3a6-4aea-9b71-0127f358e9cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:30.849998 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40182c0-c3a6-4aea-9b71-0127f358e9cd admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:30.857589 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40182c0-c3a6-4aea-9b71-0127f358e9cd admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:05:30.859342 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40182c0-c3a6-4aea-9b71-0127f358e9cd admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:05:30.867968 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 4/8] 10.222.0.44 () {58 vars in 1324 bytes} [Wed Jun 29 11:05:30 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 29 11:05:30.873021 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6613534-018e-490f-9977-761ad3404853 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:05:30.881871 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6613534-018e-490f-9977-761ad3404853 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:05:30.886739 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288b3921-9852-477d-ad22-2d75f0494e8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:05:30.887593 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288b3921-9852-477d-ad22-2d75f0494e8d admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:30.887911 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288b3921-9852-477d-ad22-2d75f0494e8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:30.888195 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288b3921-9852-477d-ad22-2d75f0494e8d admin admin] PATH_INFO: `/v3/users/cdb4139035bd4d46a35dca750b630776` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:30.889582 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288b3921-9852-477d-ad22-2d75f0494e8d admin admin] RBAC: Authorizing `identity:get_user(user_id=cdb4139035bd4d46a35dca750b630776)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:05:30.891403 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288b3921-9852-477d-ad22-2d75f0494e8d admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:05:30.892999 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 5/9] 10.222.0.44 () {56 vars in 1198 bytes} [Wed Jun 29 11:05:30 2022] GET /identity/v3/users/cdb4139035bd4d46a35dca750b630776 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:05:32.956824 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbd0dbb-0e66-43b2-a137-71c169c3a1d5 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:32.957330 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbd0dbb-0e66-43b2-a137-71c169c3a1d5 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:32.957416 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbd0dbb-0e66-43b2-a137-71c169c3a1d5 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:32.958714 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 5/10] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:05:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 29 11:05:32.963300 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09944395-e721-4b4a-a467-d66c899cebb4 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:32.963737 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d618da-8e46-4edc-8158-f4cc8f96f183 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:32.963864 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09944395-e721-4b4a-a467-d66c899cebb4 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:32.963961 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d618da-8e46-4edc-8158-f4cc8f96f183 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:32.964049 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09944395-e721-4b4a-a467-d66c899cebb4 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:32.964135 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d618da-8e46-4edc-8158-f4cc8f96f183 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:32.965238 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 6/11] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:05:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:05:32.969934 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d38571-26e1-4a0a-8028-ca7c7cf5b514 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:32.970226 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d38571-26e1-4a0a-8028-ca7c7cf5b514 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:32.970730 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d38571-26e1-4a0a-8028-ca7c7cf5b514 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:32.987054 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-a4d38571-26e1-4a0a-8028-ca7c7cf5b514 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:05:32.988459 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-a4d38571-26e1-4a0a-8028-ca7c7cf5b514 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:05:33.009910 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG passlib.handlers.bcrypt [None req-a9d618da-8e46-4edc-8158-f4cc8f96f183 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80166) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 29 11:05:33.010338 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG passlib.handlers.bcrypt [None req-a9d618da-8e46-4edc-8158-f4cc8f96f183 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80166) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 29 11:05:33.056248 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-a9d618da-8e46-4edc-8158-f4cc8f96f183 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:05:33.057819 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-a9d618da-8e46-4edc-8158-f4cc8f96f183 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:05:33.069555 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 7/12] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:05:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 29 11:05:33.091794 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d38571-26e1-4a0a-8028-ca7c7cf5b514 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:05:33.114779 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:05:33.157307 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 6/13] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:05:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 29 11:05:33.181386 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d618da-8e46-4edc-8158-f4cc8f96f183 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:05:33.200717 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d38571-26e1-4a0a-8028-ca7c7cf5b514 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:05:33.234216 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0572eb4-9d87-47b0-bd5b-a89bd1839a77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:05:33.234920 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0572eb4-9d87-47b0-bd5b-a89bd1839a77 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:33.235211 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0572eb4-9d87-47b0-bd5b-a89bd1839a77 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:33.235489 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0572eb4-9d87-47b0-bd5b-a89bd1839a77 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:33.284574 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d618da-8e46-4edc-8158-f4cc8f96f183 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:05:33.286853 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:05:33.308238 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0572eb4-9d87-47b0-bd5b-a89bd1839a77 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:05:33.320673 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0572eb4-9d87-47b0-bd5b-a89bd1839a77 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:05:33.323324 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51618baa-2cbb-41e2-b36c-dcf77b655634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:05:33.324391 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51618baa-2cbb-41e2-b36c-dcf77b655634 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:33.324682 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51618baa-2cbb-41e2-b36c-dcf77b655634 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:33.324959 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51618baa-2cbb-41e2-b36c-dcf77b655634 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:33.332921 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:05:33.336798 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51618baa-2cbb-41e2-b36c-dcf77b655634 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 11:05:33.348145 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-51618baa-2cbb-41e2-b36c-dcf77b655634 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 11:05:33.348145 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:05:33.348145 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:05:33.348145 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:05:33.348145 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:05:33.348145 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:05:33.348145 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:05:33.348145 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:05:33.348145 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:05:33.348145 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:05:33.348145 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:05:33.348145 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:05:33.348145 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 11:05:33.348145 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 11:05:33.348145 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 11:05:33.348145 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 11:05:33.349109 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 11:05:33.349109 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 11:05:33.349109 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 11:05:33.349109 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 11:05:33.349109 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:05:33.349109 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:05:33.349109 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:05:33.349109 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:05:33.349109 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:05:33.349109 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:05:33.349109 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:05:33.349109 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jun 29 11:05:33.349109 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:05:33.349109 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:05:33.349109 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:05:33.349109 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:05:33.352250 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:05:33.352250 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:05:33.352250 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:05:33.352250 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:05:33.352250 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 11:05:33.352250 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 11:05:33.352250 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 11:05:33.352250 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 11:05:33.352250 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 11:05:33.352250 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 11:05:33.352250 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 11:05:33.352250 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:05:33.352250 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 7/14] 10.222.0.44 () {56 vars in 1096 bytes} [Wed Jun 29 11:05:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 11:05:33.355507 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51618baa-2cbb-41e2-b36c-dcf77b655634 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:05:33.407184 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 8/15] 10.222.0.44 () {58 vars in 1282 bytes} [Wed Jun 29 11:05:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 29 11:05:33.412660 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0572eb4-9d87-47b0-bd5b-a89bd1839a77 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:05:33.453682 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51618baa-2cbb-41e2-b36c-dcf77b655634 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:05:33.486217 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af76bc38-eb83-4258-8f5a-dc3a005fb10e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:05:33.486881 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af76bc38-eb83-4258-8f5a-dc3a005fb10e None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:33.487176 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af76bc38-eb83-4258-8f5a-dc3a005fb10e None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:33.487456 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af76bc38-eb83-4258-8f5a-dc3a005fb10e None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:33.488255 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af76bc38-eb83-4258-8f5a-dc3a005fb10e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:05:33.488569 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af76bc38-eb83-4258-8f5a-dc3a005fb10e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:05:33.490003 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af76bc38-eb83-4258-8f5a-dc3a005fb10e None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:05:33.502141 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 8/16] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Jun 29 11:05:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 11:05:33.507732 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af76bc38-eb83-4258-8f5a-dc3a005fb10e None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:05:33.507930 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0572eb4-9d87-47b0-bd5b-a89bd1839a77 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:05:33.530658 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:05:33.535477 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:05:33.541961 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e96384-93bf-4009-a960-454e1fa6a006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:05:33.542754 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e96384-93bf-4009-a960-454e1fa6a006 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:33.544436 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e96384-93bf-4009-a960-454e1fa6a006 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:33.544436 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e96384-93bf-4009-a960-454e1fa6a006 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:33.553220 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e96384-93bf-4009-a960-454e1fa6a006 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:05:33.554693 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e96384-93bf-4009-a960-454e1fa6a006 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:05:33.576895 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-52e96384-93bf-4009-a960-454e1fa6a006 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 11:05:33.576895 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:05:33.576895 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:05:33.576895 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:05:33.576895 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:05:33.576895 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:05:33.576895 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:05:33.576895 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:05:33.576895 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:05:33.576895 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:05:33.576895 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:05:33.576895 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:05:33.576895 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 11:05:33.576895 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 11:05:33.576895 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 11:05:33.577764 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 11:05:33.577764 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:05:33.577764 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:05:33.577764 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:05:33.577764 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:05:33.577764 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:05:33.577764 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:05:33.577764 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:05:33.577764 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jun 29 11:05:33.577764 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:05:33.577764 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:05:33.577764 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:05:33.577764 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:05:33.577764 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:05:33.577764 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:05:33.577764 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:05:33.577764 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:05:33.578560 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 11:05:33.578560 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 11:05:33.578560 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 11:05:33.578560 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jun 29 11:05:33.578560 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 9/17] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jun 29 11:05:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 11:05:33.583762 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e96384-93bf-4009-a960-454e1fa6a006 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:05:33.610409 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af76bc38-eb83-4258-8f5a-dc3a005fb10e None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:05:33.643386 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ef8b9f-b2fa-4589-b55a-360e267c1b35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:05:33.644284 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ef8b9f-b2fa-4589-b55a-360e267c1b35 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:33.644574 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ef8b9f-b2fa-4589-b55a-360e267c1b35 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:33.644850 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ef8b9f-b2fa-4589-b55a-360e267c1b35 None admin] PATH_INFO: `/v3/users/cdb4139035bd4d46a35dca750b630776` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:33.646366 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ef8b9f-b2fa-4589-b55a-360e267c1b35 None admin] RBAC: Authorizing `identity:get_user(user_id=cdb4139035bd4d46a35dca750b630776)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:05:33.647801 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ef8b9f-b2fa-4589-b55a-360e267c1b35 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:05:33.649439 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 9/18] 10.222.0.44 () {56 vars in 1177 bytes} [Wed Jun 29 11:05:33 2022] GET /identity/v3/users/cdb4139035bd4d46a35dca750b630776 => generated 247 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 11:05:33.654673 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ef8b9f-b2fa-4589-b55a-360e267c1b35 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:05:33.682431 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e96384-93bf-4009-a960-454e1fa6a006 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:05:33.716410 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb096da-5437-4428-9d80-d39b14be6e25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:05:33.717004 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb096da-5437-4428-9d80-d39b14be6e25 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:33.717290 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb096da-5437-4428-9d80-d39b14be6e25 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:33.717567 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb096da-5437-4428-9d80-d39b14be6e25 None admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:33.718288 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb096da-5437-4428-9d80-d39b14be6e25 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:05:33.718597 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb096da-5437-4428-9d80-d39b14be6e25 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:05:33.720016 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb096da-5437-4428-9d80-d39b14be6e25 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:05:33.736566 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 10/19] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Jun 29 11:05:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 11:05:33.747527 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:05:33.754588 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ef8b9f-b2fa-4589-b55a-360e267c1b35 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:05:33.791396 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebc4f05-45dc-447b-8a78-504b550d5469 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:05:33.792255 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebc4f05-45dc-447b-8a78-504b550d5469 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:33.792539 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebc4f05-45dc-447b-8a78-504b550d5469 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:33.792813 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebc4f05-45dc-447b-8a78-504b550d5469 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:33.794052 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebc4f05-45dc-447b-8a78-504b550d5469 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:05:33.796405 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebc4f05-45dc-447b-8a78-504b550d5469 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:05:33.798285 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 10/20] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jun 29 11:05:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 11:05:33.804135 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb096da-5437-4428-9d80-d39b14be6e25 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:05:33.839945 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:05:33.909612 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb096da-5437-4428-9d80-d39b14be6e25 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:05:33.950908 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f762fb7-f068-48cc-81de-f4e68ba0ad7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:05:33.951891 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f762fb7-f068-48cc-81de-f4e68ba0ad7a None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:33.952159 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f762fb7-f068-48cc-81de-f4e68ba0ad7a None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:33.952438 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f762fb7-f068-48cc-81de-f4e68ba0ad7a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:33.953282 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f762fb7-f068-48cc-81de-f4e68ba0ad7a None admin] RBAC: Adding query filter params (role.id=e95c2ce63e114cc19a9dda6ad21e794f, scope.domain.id=default, user.id=cdb4139035bd4d46a35dca750b630776) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:05:33.953599 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f762fb7-f068-48cc-81de-f4e68ba0ad7a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:05:33.955059 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f762fb7-f068-48cc-81de-f4e68ba0ad7a None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:05:33.962845 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 11/21] 10.222.0.44 () {56 vars in 1322 bytes} [Wed Jun 29 11:05:33 2022] GET /identity/v3/role_assignments?user.id=cdb4139035bd4d46a35dca750b630776&scope.domain.id=default&role.id=e95c2ce63e114cc19a9dda6ad21e794f => generated 233 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:05:36.059908 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d7edf1-896f-4ccf-b629-434f9324f467 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:36.060939 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d7edf1-896f-4ccf-b629-434f9324f467 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:36.061428 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d7edf1-896f-4ccf-b629-434f9324f467 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:36.063028 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 11/22] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:05:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:05:36.068097 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca98b2f2-d3cb-4204-81ab-7435628a36cd None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:36.068921 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca98b2f2-d3cb-4204-81ab-7435628a36cd None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:36.069409 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca98b2f2-d3cb-4204-81ab-7435628a36cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:36.096800 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-ca98b2f2-d3cb-4204-81ab-7435628a36cd None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:05:36.098333 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-ca98b2f2-d3cb-4204-81ab-7435628a36cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:05:36.187366 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 12/23] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:05:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 11:05:36.222042 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d7edf1-896f-4ccf-b629-434f9324f467 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:05:36.367067 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d7edf1-896f-4ccf-b629-434f9324f467 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:05:36.410822 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e552f9-ae11-4560-8b98-9f7ef927d8b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:05:36.412002 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e552f9-ae11-4560-8b98-9f7ef927d8b9 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:36.412519 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e552f9-ae11-4560-8b98-9f7ef927d8b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:36.412981 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e552f9-ae11-4560-8b98-9f7ef927d8b9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:36.438536 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e552f9-ae11-4560-8b98-9f7ef927d8b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:05:36.439961 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e552f9-ae11-4560-8b98-9f7ef927d8b9 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:05:36.464384 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-96e552f9-ae11-4560-8b98-9f7ef927d8b9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 11:05:36.464384 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:05:36.464384 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:05:36.464384 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:05:36.464384 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:05:36.464384 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:05:36.464384 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:05:36.464384 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:05:36.464384 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:05:36.464384 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:05:36.464384 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:05:36.464384 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:05:36.464384 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 11:05:36.464384 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 11:05:36.464384 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 11:05:36.465225 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 11:05:36.465225 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:05:36.465225 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:05:36.465225 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:05:36.465225 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:05:36.465225 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:05:36.465225 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:05:36.465225 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:05:36.465225 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jun 29 11:05:36.465225 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:05:36.465225 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:05:36.465225 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:05:36.465225 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:05:36.465225 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:05:36.465225 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:05:36.465225 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:05:36.465225 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:05:36.466009 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 11:05:36.466009 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 11:05:36.466009 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 11:05:36.466009 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:05:36.466009 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 12/24] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jun 29 11:05:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:05:36.472374 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca98b2f2-d3cb-4204-81ab-7435628a36cd None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:05:36.525406 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:05:36.571542 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637c5d4e-aca7-48c7-bb95-a7fe8356f762 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:36.571838 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637c5d4e-aca7-48c7-bb95-a7fe8356f762 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:36.572125 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637c5d4e-aca7-48c7-bb95-a7fe8356f762 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:36.573738 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 13/25] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:05:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 29 11:05:36.590017 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05b2ba3-11b3-4a70-8230-64e4f1b31763 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:36.590519 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05b2ba3-11b3-4a70-8230-64e4f1b31763 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:36.590954 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05b2ba3-11b3-4a70-8230-64e4f1b31763 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:36.624851 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-e05b2ba3-11b3-4a70-8230-64e4f1b31763 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:05:36.626315 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-e05b2ba3-11b3-4a70-8230-64e4f1b31763 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:05:36.629103 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca98b2f2-d3cb-4204-81ab-7435628a36cd None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:05:36.643482 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:05:36.693825 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd8f760-1746-4aea-bb3a-542da734f357 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:05:36.694573 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd8f760-1746-4aea-bb3a-542da734f357 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:36.694950 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd8f760-1746-4aea-bb3a-542da734f357 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:36.695322 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd8f760-1746-4aea-bb3a-542da734f357 None admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:36.697802 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd8f760-1746-4aea-bb3a-542da734f357 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:05:36.697802 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd8f760-1746-4aea-bb3a-542da734f357 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:05:36.699011 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd8f760-1746-4aea-bb3a-542da734f357 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:05:36.716506 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 13/26] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Jun 29 11:05:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:05:36.722328 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aae9c38-c196-4c9b-a21d-a9e200bf6676 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:36.722951 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aae9c38-c196-4c9b-a21d-a9e200bf6676 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:36.723456 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aae9c38-c196-4c9b-a21d-a9e200bf6676 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:36.724986 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 14/27] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:05:36 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:05:36.725877 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aae9c38-c196-4c9b-a21d-a9e200bf6676 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:05:36.770265 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 14/28] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:05:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 29 11:05:36.771884 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05c5bf6-ff6f-490d-a05a-7f01cd9a951c None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:36.772384 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05c5bf6-ff6f-490d-a05a-7f01cd9a951c None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:36.772880 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05c5bf6-ff6f-490d-a05a-7f01cd9a951c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:36.801043 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-b05c5bf6-ff6f-490d-a05a-7f01cd9a951c None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:05:36.802266 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-b05c5bf6-ff6f-490d-a05a-7f01cd9a951c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:05:36.858583 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:05:36.876247 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aae9c38-c196-4c9b-a21d-a9e200bf6676 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:05:36.887421 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 15/29] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:05:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 11:05:36.889077 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042f3396-0b12-4761-92e3-331c3e41264a None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:36.889539 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042f3396-0b12-4761-92e3-331c3e41264a None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:36.889987 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042f3396-0b12-4761-92e3-331c3e41264a None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:36.891608 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 16/30] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:05:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:05:36.892501 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042f3396-0b12-4761-92e3-331c3e41264a None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:05:36.942746 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f6e463-43c9-4d1b-947d-10a9c5168c9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:05:36.944597 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f6e463-43c9-4d1b-947d-10a9c5168c9f None admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:36.945053 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f6e463-43c9-4d1b-947d-10a9c5168c9f None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:36.946057 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f6e463-43c9-4d1b-947d-10a9c5168c9f None admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:36.947941 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f6e463-43c9-4d1b-947d-10a9c5168c9f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:05:36.949905 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f6e463-43c9-4d1b-947d-10a9c5168c9f None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:05:37.006330 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:05:37.033610 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042f3396-0b12-4761-92e3-331c3e41264a None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:05:37.071873 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538b68f7-13b7-43f6-af6c-51e862cedff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:05:37.072930 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538b68f7-13b7-43f6-af6c-51e862cedff9 None admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:37.073184 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538b68f7-13b7-43f6-af6c-51e862cedff9 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:37.073689 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538b68f7-13b7-43f6-af6c-51e862cedff9 None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:37.074501 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538b68f7-13b7-43f6-af6c-51e862cedff9 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:05:37.075944 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538b68f7-13b7-43f6-af6c-51e862cedff9 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:05:37.180725 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 15/31] 10.222.0.44 () {60 vars in 1141 bytes} [Wed Jun 29 11:05:36 2022] POST /identity/v3/projects => generated 301 bytes in 455 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:05:37.183070 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea77c57-13ac-43b9-a41e-7ce1fecd0bda None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:37.183599 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea77c57-13ac-43b9-a41e-7ce1fecd0bda None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:37.184101 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea77c57-13ac-43b9-a41e-7ce1fecd0bda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:37.218935 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-8ea77c57-13ac-43b9-a41e-7ce1fecd0bda None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:05:37.220824 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-8ea77c57-13ac-43b9-a41e-7ce1fecd0bda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:05:37.275934 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 17/32] 10.222.0.44 () {60 vars in 1131 bytes} [Wed Jun 29 11:05:36 2022] POST /identity/v3/roles => generated 219 bytes in 378 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:05:37.275934 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538b68f7-13b7-43f6-af6c-51e862cedff9 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:05:37.422645 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 16/33] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:05:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 11:05:37.424079 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79c7584-4055-4177-a256-ff1579478462 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:37.424374 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79c7584-4055-4177-a256-ff1579478462 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:37.424774 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79c7584-4055-4177-a256-ff1579478462 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:37.426155 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 17/34] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:05:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:05:37.432373 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dca4d3b-6413-4167-8d71-24b8c8f12e44 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:37.432660 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dca4d3b-6413-4167-8d71-24b8c8f12e44 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:37.432942 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dca4d3b-6413-4167-8d71-24b8c8f12e44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:37.463624 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538b68f7-13b7-43f6-af6c-51e862cedff9 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:05:37.470117 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-9dca4d3b-6413-4167-8d71-24b8c8f12e44 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:05:37.470571 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-9dca4d3b-6413-4167-8d71-24b8c8f12e44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:05:37.509690 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3b8558-e908-4f99-b84b-d022ddb2bd23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:05:37.510509 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3b8558-e908-4f99-b84b-d022ddb2bd23 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:37.510798 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3b8558-e908-4f99-b84b-d022ddb2bd23 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:37.511077 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3b8558-e908-4f99-b84b-d022ddb2bd23 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:37.516172 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:05:37.526650 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3b8558-e908-4f99-b84b-d022ddb2bd23 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 11:05:37.536348 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-9b3b8558-e908-4f99-b84b-d022ddb2bd23 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 11:05:37.536348 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:05:37.536348 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:05:37.536348 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:05:37.536348 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:05:37.536348 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:05:37.536348 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:05:37.536348 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:05:37.536348 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:05:37.536348 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:05:37.536348 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:05:37.536348 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:05:37.536348 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 11:05:37.536348 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 11:05:37.536348 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 11:05:37.536348 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 11:05:37.537196 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 11:05:37.537196 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 11:05:37.537196 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 11:05:37.537196 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 11:05:37.537196 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:05:37.537196 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:05:37.537196 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:05:37.537196 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:05:37.537196 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:05:37.537196 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:05:37.537196 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:05:37.537196 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jun 29 11:05:37.537196 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:05:37.537196 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:05:37.537196 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:05:37.537196 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:05:37.537956 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:05:37.537956 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:05:37.537956 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:05:37.537956 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:05:37.537956 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 11:05:37.537956 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 11:05:37.537956 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 11:05:37.537956 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 11:05:37.537956 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 11:05:37.537956 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 11:05:37.537956 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 11:05:37.537956 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:05:37.537956 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 18/35] 10.222.0.44 () {56 vars in 1096 bytes} [Wed Jun 29 11:05:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:05:37.539766 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3b8558-e908-4f99-b84b-d022ddb2bd23 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:05:37.558917 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:05:37.564715 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 18/36] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:05:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 11:05:37.566652 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dca4d3b-6413-4167-8d71-24b8c8f12e44 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:05:37.677559 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3b8558-e908-4f99-b84b-d022ddb2bd23 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:05:37.693941 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dca4d3b-6413-4167-8d71-24b8c8f12e44 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:05:37.715928 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec78b8ef-2a2e-46de-aa35-57a5e8147e03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:05:37.716609 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec78b8ef-2a2e-46de-aa35-57a5e8147e03 None admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:37.716901 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec78b8ef-2a2e-46de-aa35-57a5e8147e03 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:37.717184 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec78b8ef-2a2e-46de-aa35-57a5e8147e03 None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:37.718052 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec78b8ef-2a2e-46de-aa35-57a5e8147e03 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:05:37.719699 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec78b8ef-2a2e-46de-aa35-57a5e8147e03 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:05:37.731375 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cbb2b9-804f-4b96-95e0-da416ca2c840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:05:37.731375 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cbb2b9-804f-4b96-95e0-da416ca2c840 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:37.731375 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cbb2b9-804f-4b96-95e0-da416ca2c840 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:37.731375 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cbb2b9-804f-4b96-95e0-da416ca2c840 None admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:37.731727 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cbb2b9-804f-4b96-95e0-da416ca2c840 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:05:37.731727 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cbb2b9-804f-4b96-95e0-da416ca2c840 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:05:37.732604 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cbb2b9-804f-4b96-95e0-da416ca2c840 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:05:37.744603 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 19/37] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Jun 29 11:05:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:05:37.745359 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cbb2b9-804f-4b96-95e0-da416ca2c840 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:05:37.783734 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 19/38] 10.222.0.44 () {60 vars in 1131 bytes} [Wed Jun 29 11:05:37 2022] POST /identity/v3/roles => generated 225 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:05:37.785182 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec78b8ef-2a2e-46de-aa35-57a5e8147e03 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:05:37.840440 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:05:37.859135 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:05:37.895314 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cbb2b9-804f-4b96-95e0-da416ca2c840 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:05:37.925113 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec78b8ef-2a2e-46de-aa35-57a5e8147e03 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:05:37.942475 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7c158a-7c1d-4830-80e0-146b463d7073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:05:37.943405 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7c158a-7c1d-4830-80e0-146b463d7073 None admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:37.943689 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7c158a-7c1d-4830-80e0-146b463d7073 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:37.943999 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7c158a-7c1d-4830-80e0-146b463d7073 None admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:37.944756 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7c158a-7c1d-4830-80e0-146b463d7073 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:05:37.946720 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7c158a-7c1d-4830-80e0-146b463d7073 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:05:37.967527 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8fc026-e1a5-4d9b-87aa-3f8da842ac1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:05:37.968436 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8fc026-e1a5-4d9b-87aa-3f8da842ac1b None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:37.968723 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8fc026-e1a5-4d9b-87aa-3f8da842ac1b None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:37.969001 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8fc026-e1a5-4d9b-87aa-3f8da842ac1b None admin] PATH_INFO: `/v3/users/cdb4139035bd4d46a35dca750b630776` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:37.970326 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8fc026-e1a5-4d9b-87aa-3f8da842ac1b None admin] RBAC: Authorizing `identity:get_user(user_id=cdb4139035bd4d46a35dca750b630776)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:05:37.971799 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8fc026-e1a5-4d9b-87aa-3f8da842ac1b None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:05:37.973596 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 20/39] 10.222.0.44 () {56 vars in 1177 bytes} [Wed Jun 29 11:05:37 2022] GET /identity/v3/users/cdb4139035bd4d46a35dca750b630776 => generated 247 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:05:37.974859 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec62a3f-571b-4ec4-ba5b-3d6e5ce968e2 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:37.975144 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec62a3f-571b-4ec4-ba5b-3d6e5ce968e2 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:37.975419 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec62a3f-571b-4ec4-ba5b-3d6e5ce968e2 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:37.976903 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 21/40] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:05:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:05:37.979176 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec62a3f-571b-4ec4-ba5b-3d6e5ce968e2 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:05:38.023551 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 20/41] 10.222.0.44 () {60 vars in 1131 bytes} [Wed Jun 29 11:05:37 2022] POST /identity/v3/roles => generated 223 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:05:38.024802 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b78f9f-a10b-48b8-950d-09e3d9eece6c None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:38.025093 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b78f9f-a10b-48b8-950d-09e3d9eece6c None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:38.025373 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b78f9f-a10b-48b8-950d-09e3d9eece6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:38.103903 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec62a3f-571b-4ec4-ba5b-3d6e5ce968e2 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:05:38.131141 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:05:38.144265 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834769de-5f1a-41fe-b2b9-4527c1c160db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:05:38.145097 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834769de-5f1a-41fe-b2b9-4527c1c160db None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:38.145387 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834769de-5f1a-41fe-b2b9-4527c1c160db None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:38.145667 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834769de-5f1a-41fe-b2b9-4527c1c160db None admin] PATH_INFO: `/v3/domains/default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:38.147028 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834769de-5f1a-41fe-b2b9-4527c1c160db None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:05:38.148587 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834769de-5f1a-41fe-b2b9-4527c1c160db None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:05:38.150521 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 22/42] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jun 29 11:05:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 11:05:38.152377 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-09b78f9f-a10b-48b8-950d-09e3d9eece6c None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:05:38.153640 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-09b78f9f-a10b-48b8-950d-09e3d9eece6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:05:38.156427 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834769de-5f1a-41fe-b2b9-4527c1c160db None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:05:38.232957 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 21/43] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:05:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 11:05:38.255792 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b78f9f-a10b-48b8-950d-09e3d9eece6c None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:05:38.277906 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834769de-5f1a-41fe-b2b9-4527c1c160db None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:05:38.307075 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:05:38.311354 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeac5c9e-86da-4dd1-8306-334fb094127e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:05:38.312265 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeac5c9e-86da-4dd1-8306-334fb094127e None admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:38.312553 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeac5c9e-86da-4dd1-8306-334fb094127e None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:38.312834 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeac5c9e-86da-4dd1-8306-334fb094127e None admin] PATH_INFO: `/v3/domains/default/users/cdb4139035bd4d46a35dca750b630776/roles/e95c2ce63e114cc19a9dda6ad21e794f` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:38.314847 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeac5c9e-86da-4dd1-8306-334fb094127e None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=cdb4139035bd4d46a35dca750b630776, role_id=e95c2ce63e114cc19a9dda6ad21e794f)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:05:38.316330 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeac5c9e-86da-4dd1-8306-334fb094127e None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:05:38.353391 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b78f9f-a10b-48b8-950d-09e3d9eece6c None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:05:38.385578 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efbe11a-0a46-46fa-a020-89b014f3d8bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:05:38.386364 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efbe11a-0a46-46fa-a020-89b014f3d8bc None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:38.386651 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efbe11a-0a46-46fa-a020-89b014f3d8bc None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:38.386929 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efbe11a-0a46-46fa-a020-89b014f3d8bc None admin] PATH_INFO: `/v3/domains/default` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:38.388171 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efbe11a-0a46-46fa-a020-89b014f3d8bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:05:38.389620 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efbe11a-0a46-46fa-a020-89b014f3d8bc None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:05:38.391218 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 22/44] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jun 29 11:05:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 11:05:38.396734 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efbe11a-0a46-46fa-a020-89b014f3d8bc None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:05:38.457140 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 23/45] 10.222.0.44 () {58 vars in 1361 bytes} [Wed Jun 29 11:05:38 2022] PUT /identity/v3/domains/default/users/cdb4139035bd4d46a35dca750b630776/roles/e95c2ce63e114cc19a9dda6ad21e794f => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 11:05:38.497868 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efbe11a-0a46-46fa-a020-89b014f3d8bc None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:05:38.519613 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:05:38.535269 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce763b0-bf45-40d7-b1b3-c3c7d79b88af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:05:38.535269 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce763b0-bf45-40d7-b1b3-c3c7d79b88af None admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:38.535269 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce763b0-bf45-40d7-b1b3-c3c7d79b88af None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:38.535576 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce763b0-bf45-40d7-b1b3-c3c7d79b88af None admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:38.536190 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce763b0-bf45-40d7-b1b3-c3c7d79b88af None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:05:38.537687 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce763b0-bf45-40d7-b1b3-c3c7d79b88af None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:05:38.619500 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 23/46] 10.222.0.44 () {60 vars in 1141 bytes} [Wed Jun 29 11:05:38 2022] POST /identity/v3/projects => generated 312 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 11:05:40.464915 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a48441-7cb1-4acf-a9b3-98a92be63d62 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:40.465485 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a48441-7cb1-4acf-a9b3-98a92be63d62 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:40.465485 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a48441-7cb1-4acf-a9b3-98a92be63d62 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:40.466850 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 24/47] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:05:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:05:40.471877 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48637c0-92db-42d6-9a9f-103782dbf476 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:40.472244 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48637c0-92db-42d6-9a9f-103782dbf476 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:40.472587 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48637c0-92db-42d6-9a9f-103782dbf476 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:40.488841 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-b48637c0-92db-42d6-9a9f-103782dbf476 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:05:40.490022 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-b48637c0-92db-42d6-9a9f-103782dbf476 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:05:40.554182 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 24/48] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:05:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 11:05:40.578088 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a48441-7cb1-4acf-a9b3-98a92be63d62 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:05:40.628239 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:05:40.664130 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e619ea-408d-484b-adc8-56895bf14dbc None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:40.664423 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e619ea-408d-484b-adc8-56895bf14dbc None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:40.664703 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e619ea-408d-484b-adc8-56895bf14dbc None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:40.666240 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 25/49] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:05:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 29 11:05:40.670937 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd76d02-3003-4aed-8979-4b6f3417e93c None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:40.671229 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd76d02-3003-4aed-8979-4b6f3417e93c None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:40.671524 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd76d02-3003-4aed-8979-4b6f3417e93c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:40.678384 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a48441-7cb1-4acf-a9b3-98a92be63d62 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:05:40.688297 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-fcd76d02-3003-4aed-8979-4b6f3417e93c None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:05:40.689626 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-fcd76d02-3003-4aed-8979-4b6f3417e93c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:05:40.714073 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52361517-8264-40e2-9c2d-d57ab685796a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:05:40.714861 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52361517-8264-40e2-9c2d-d57ab685796a None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:40.715207 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52361517-8264-40e2-9c2d-d57ab685796a None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:40.715425 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52361517-8264-40e2-9c2d-d57ab685796a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:40.725243 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52361517-8264-40e2-9c2d-d57ab685796a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 11:05:40.732810 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-52361517-8264-40e2-9c2d-d57ab685796a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 11:05:40.732810 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:05:40.732810 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:05:40.732810 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:05:40.732810 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:05:40.732810 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:05:40.732810 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:05:40.732810 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:05:40.732810 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:05:40.732810 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:05:40.732810 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:05:40.732810 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:05:40.732810 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 11:05:40.732810 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 11:05:40.732810 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 11:05:40.732810 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 11:05:40.733774 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 11:05:40.733774 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 11:05:40.733774 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 11:05:40.733774 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 11:05:40.733774 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:05:40.733774 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:05:40.733774 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:05:40.733774 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:05:40.733774 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:05:40.733774 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:05:40.733774 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:05:40.733774 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jun 29 11:05:40.733774 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:05:40.733774 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:05:40.733774 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:05:40.733774 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:05:40.734525 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:05:40.734525 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:05:40.734525 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:05:40.734525 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:05:40.734525 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 11:05:40.734525 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 11:05:40.734525 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 11:05:40.734525 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 11:05:40.734525 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 11:05:40.734525 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 11:05:40.734525 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 11:05:40.734525 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:05:40.734525 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 25/50] 10.222.0.44 () {56 vars in 1096 bytes} [Wed Jun 29 11:05:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:05:40.739660 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52361517-8264-40e2-9c2d-d57ab685796a None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:05:40.758408 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 26/51] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:05:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 29 11:05:40.781517 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd76d02-3003-4aed-8979-4b6f3417e93c None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:05:40.811490 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:05:40.837900 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52361517-8264-40e2-9c2d-d57ab685796a None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:05:40.848184 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:05:40.870994 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7cd568-1708-49bf-9697-773829527fa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:05:40.871756 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7cd568-1708-49bf-9697-773829527fa7 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:40.872083 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7cd568-1708-49bf-9697-773829527fa7 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:40.872365 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7cd568-1708-49bf-9697-773829527fa7 None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:40.873208 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7cd568-1708-49bf-9697-773829527fa7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:05:40.873521 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7cd568-1708-49bf-9697-773829527fa7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:05:40.874905 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7cd568-1708-49bf-9697-773829527fa7 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:05:40.879592 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd76d02-3003-4aed-8979-4b6f3417e93c None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:05:40.883842 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 26/52] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Jun 29 11:05:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 11:05:40.889282 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7cd568-1708-49bf-9697-773829527fa7 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:05:40.913864 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b3b883-9682-465e-804a-26bd8c55402a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:05:40.914818 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b3b883-9682-465e-804a-26bd8c55402a None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:40.915212 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b3b883-9682-465e-804a-26bd8c55402a None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:40.915594 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b3b883-9682-465e-804a-26bd8c55402a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:40.916903 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b3b883-9682-465e-804a-26bd8c55402a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:05:40.918365 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b3b883-9682-465e-804a-26bd8c55402a None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:05:40.920006 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 27/53] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jun 29 11:05:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 11:05:40.925316 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b3b883-9682-465e-804a-26bd8c55402a None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:05:40.989396 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7cd568-1708-49bf-9697-773829527fa7 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:05:41.023163 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:05:41.023382 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd600128-0030-4405-b8ba-162d896ebd5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:05:41.024285 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd600128-0030-4405-b8ba-162d896ebd5d None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:41.024575 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd600128-0030-4405-b8ba-162d896ebd5d None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:41.024856 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd600128-0030-4405-b8ba-162d896ebd5d None admin] PATH_INFO: `/v3/users/cdb4139035bd4d46a35dca750b630776` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:41.025977 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b3b883-9682-465e-804a-26bd8c55402a None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:05:41.026067 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd600128-0030-4405-b8ba-162d896ebd5d None admin] RBAC: Authorizing `identity:get_user(user_id=cdb4139035bd4d46a35dca750b630776)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:05:41.027487 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd600128-0030-4405-b8ba-162d896ebd5d None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:05:41.029258 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 27/54] 10.222.0.44 () {56 vars in 1177 bytes} [Wed Jun 29 11:05:40 2022] GET /identity/v3/users/cdb4139035bd4d46a35dca750b630776 => generated 247 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 11:05:41.034488 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd600128-0030-4405-b8ba-162d896ebd5d None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:05:41.059598 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058b34c5-4ce8-418f-ba4a-0c3a5c9b0062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:05:41.060298 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058b34c5-4ce8-418f-ba4a-0c3a5c9b0062 None admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:41.060587 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058b34c5-4ce8-418f-ba4a-0c3a5c9b0062 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:41.060868 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058b34c5-4ce8-418f-ba4a-0c3a5c9b0062 None admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:41.061577 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058b34c5-4ce8-418f-ba4a-0c3a5c9b0062 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:05:41.063274 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058b34c5-4ce8-418f-ba4a-0c3a5c9b0062 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:05:41.155762 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 28/55] 10.222.0.44 () {60 vars in 1140 bytes} [Wed Jun 29 11:05:40 2022] POST /identity/v3/projects => generated 298 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 11:05:41.179604 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:05:41.243573 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd600128-0030-4405-b8ba-162d896ebd5d None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:05:41.279518 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e32f7e-949e-481b-95eb-99c13658f2fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:05:41.280367 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e32f7e-949e-481b-95eb-99c13658f2fa None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:41.280656 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e32f7e-949e-481b-95eb-99c13658f2fa None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:41.280936 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e32f7e-949e-481b-95eb-99c13658f2fa None admin] PATH_INFO: `/v3/domains/default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:41.282161 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e32f7e-949e-481b-95eb-99c13658f2fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:05:41.283629 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e32f7e-949e-481b-95eb-99c13658f2fa None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:05:41.285346 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 28/56] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jun 29 11:05:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 11:05:41.293981 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058b34c5-4ce8-418f-ba4a-0c3a5c9b0062 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:05:41.400669 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058b34c5-4ce8-418f-ba4a-0c3a5c9b0062 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:05:41.433159 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207c49d4-9636-49cf-852a-e5087e28c4a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:05:41.433816 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207c49d4-9636-49cf-852a-e5087e28c4a2 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:41.434105 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207c49d4-9636-49cf-852a-e5087e28c4a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:41.434386 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207c49d4-9636-49cf-852a-e5087e28c4a2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:41.435223 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207c49d4-9636-49cf-852a-e5087e28c4a2 None admin] RBAC: Adding query filter params (role.id=e95c2ce63e114cc19a9dda6ad21e794f, scope.domain.id=default, user.id=cdb4139035bd4d46a35dca750b630776) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:05:41.435542 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207c49d4-9636-49cf-852a-e5087e28c4a2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:05:41.437024 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207c49d4-9636-49cf-852a-e5087e28c4a2 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:05:41.443071 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 29/57] 10.222.0.44 () {56 vars in 1322 bytes} [Wed Jun 29 11:05:41 2022] GET /identity/v3/role_assignments?user.id=cdb4139035bd4d46a35dca750b630776&scope.domain.id=default&role.id=e95c2ce63e114cc19a9dda6ad21e794f => generated 530 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:05:43.191665 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c825a3ec-6c0b-4118-82db-2d340f60a159 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:43.192187 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c825a3ec-6c0b-4118-82db-2d340f60a159 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:43.192267 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c825a3ec-6c0b-4118-82db-2d340f60a159 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:43.193612 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 29/58] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:05:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:05:43.199408 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3c1c67-d947-4b25-a6f9-f1ed559001da None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:43.199855 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3c1c67-d947-4b25-a6f9-f1ed559001da None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:43.199993 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3c1c67-d947-4b25-a6f9-f1ed559001da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:43.216350 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-6a3c1c67-d947-4b25-a6f9-f1ed559001da None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:05:43.217548 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-6a3c1c67-d947-4b25-a6f9-f1ed559001da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:05:43.227835 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:05:43.281818 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 30/59] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:05:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 11:05:43.306797 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c825a3ec-6c0b-4118-82db-2d340f60a159 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:05:43.406115 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c825a3ec-6c0b-4118-82db-2d340f60a159 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:05:43.415390 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:05:43.439676 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9ebfd5-62da-428b-9351-63e5cc00de61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:05:43.441241 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9ebfd5-62da-428b-9351-63e5cc00de61 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:43.441862 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9ebfd5-62da-428b-9351-63e5cc00de61 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:43.442099 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9ebfd5-62da-428b-9351-63e5cc00de61 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:43.443293 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9ebfd5-62da-428b-9351-63e5cc00de61 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:05:43.444766 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9ebfd5-62da-428b-9351-63e5cc00de61 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:05:43.446405 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 30/60] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jun 29 11:05:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:05:43.452209 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3c1c67-d947-4b25-a6f9-f1ed559001da None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:05:43.546386 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3c1c67-d947-4b25-a6f9-f1ed559001da None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:05:43.578746 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5380701f-663b-4c79-a721-af7394aff1f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:05:43.579398 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5380701f-663b-4c79-a721-af7394aff1f2 None admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:43.579679 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5380701f-663b-4c79-a721-af7394aff1f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:43.579986 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5380701f-663b-4c79-a721-af7394aff1f2 None admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:43.580682 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5380701f-663b-4c79-a721-af7394aff1f2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:05:43.582132 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5380701f-663b-4c79-a721-af7394aff1f2 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:05:43.661922 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 31/61] 10.222.0.44 () {60 vars in 1132 bytes} [Wed Jun 29 11:05:43 2022] POST /identity/v3/users => generated 275 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:05:46.754095 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ee1427-356a-4ef6-bee5-5fcb4a43b7d4 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:46.754961 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ee1427-356a-4ef6-bee5-5fcb4a43b7d4 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:46.755438 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ee1427-356a-4ef6-bee5-5fcb4a43b7d4 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:46.756998 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 31/62] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:05:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:05:46.762786 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc02d196-82eb-4691-8fe1-2492ac9c9820 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:46.763569 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc02d196-82eb-4691-8fe1-2492ac9c9820 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:46.764022 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc02d196-82eb-4691-8fe1-2492ac9c9820 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:46.769313 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:05:46.784813 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-bc02d196-82eb-4691-8fe1-2492ac9c9820 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:05:46.788349 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-bc02d196-82eb-4691-8fe1-2492ac9c9820 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:05:46.880462 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 32/63] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:05:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 29 11:05:46.920446 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ee1427-356a-4ef6-bee5-5fcb4a43b7d4 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:05:46.955300 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a23016-2a47-4b3b-9c45-e53d1576c009 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:46.955629 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a23016-2a47-4b3b-9c45-e53d1576c009 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:46.955911 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a23016-2a47-4b3b-9c45-e53d1576c009 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:46.957430 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 33/64] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:05:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:05:46.958378 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdc992a-d83b-4a22-8fa0-dbc9b7115db6 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:46.958667 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdc992a-d83b-4a22-8fa0-dbc9b7115db6 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:46.958956 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdc992a-d83b-4a22-8fa0-dbc9b7115db6 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:46.961060 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 34/65] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:05:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:05:46.966252 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36886ce4-3162-4c78-9cb2-115cfe191adb None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:46.966714 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36886ce4-3162-4c78-9cb2-115cfe191adb None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:46.967140 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36886ce4-3162-4c78-9cb2-115cfe191adb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:46.985464 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-36886ce4-3162-4c78-9cb2-115cfe191adb None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:05:46.987157 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-36886ce4-3162-4c78-9cb2-115cfe191adb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:05:47.054667 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ee1427-356a-4ef6-bee5-5fcb4a43b7d4 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:05:47.073508 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 35/66] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:05:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 29 11:05:47.075094 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fdbbfe-aaf2-43ad-afe3-095cbe3d2c8e None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:47.075527 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fdbbfe-aaf2-43ad-afe3-095cbe3d2c8e None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:47.076013 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fdbbfe-aaf2-43ad-afe3-095cbe3d2c8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:47.093228 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-07fdbbfe-aaf2-43ad-afe3-095cbe3d2c8e None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:05:47.094657 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-07fdbbfe-aaf2-43ad-afe3-095cbe3d2c8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:05:47.122961 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6273242-ba05-4a58-96d9-a3a1916113d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:05:47.124106 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6273242-ba05-4a58-96d9-a3a1916113d1 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:47.124567 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6273242-ba05-4a58-96d9-a3a1916113d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:47.125011 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6273242-ba05-4a58-96d9-a3a1916113d1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:47.137477 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6273242-ba05-4a58-96d9-a3a1916113d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 11:05:47.145951 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-d6273242-ba05-4a58-96d9-a3a1916113d1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 29 11:05:47.145951 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:05:47.145951 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:05:47.145951 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:05:47.145951 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:05:47.145951 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:05:47.145951 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:05:47.145951 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:05:47.145951 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:05:47.145951 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:05:47.145951 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:05:47.145951 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:05:47.145951 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 11:05:47.145951 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 11:05:47.145951 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 11:05:47.145951 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 11:05:47.146803 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 11:05:47.146803 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 11:05:47.146803 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 11:05:47.146803 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 11:05:47.146803 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:05:47.146803 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:05:47.146803 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:05:47.146803 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:05:47.146803 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:05:47.146803 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:05:47.146803 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:05:47.146803 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jun 29 11:05:47.146803 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:05:47.146803 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:05:47.146803 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:05:47.146803 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:05:47.147574 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:05:47.147574 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:05:47.147574 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:05:47.147574 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:05:47.147574 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 11:05:47.147574 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 11:05:47.147574 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 11:05:47.147574 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 11:05:47.147574 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 11:05:47.147574 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 11:05:47.147574 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 29 11:05:47.147574 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:05:47.150820 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 32/67] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Jun 29 11:05:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:05:47.151919 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6273242-ba05-4a58-96d9-a3a1916113d1 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:05:47.163313 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:05:47.182344 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 36/68] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:05:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 11:05:47.187956 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fdbbfe-aaf2-43ad-afe3-095cbe3d2c8e None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:05:47.206558 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:05:47.276356 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6273242-ba05-4a58-96d9-a3a1916113d1 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:05:47.311861 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fdbbfe-aaf2-43ad-afe3-095cbe3d2c8e None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:05:47.312750 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54064ac9-ed60-4b08-84a8-ec6905a807a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:05:47.313552 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54064ac9-ed60-4b08-84a8-ec6905a807a9 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:47.313831 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54064ac9-ed60-4b08-84a8-ec6905a807a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:47.314117 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54064ac9-ed60-4b08-84a8-ec6905a807a9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:47.325309 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54064ac9-ed60-4b08-84a8-ec6905a807a9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 11:05:47.333107 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-54064ac9-ed60-4b08-84a8-ec6905a807a9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 11:05:47.333107 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:05:47.333107 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:05:47.333107 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:05:47.333107 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:05:47.333107 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:05:47.333107 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:05:47.333107 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:05:47.333107 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:05:47.333107 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:05:47.333107 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:05:47.333107 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:05:47.333107 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 11:05:47.333107 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 11:05:47.333107 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 11:05:47.333107 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 11:05:47.333953 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 11:05:47.333953 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 11:05:47.333953 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 11:05:47.333953 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 11:05:47.333953 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:05:47.333953 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:05:47.333953 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:05:47.333953 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:05:47.333953 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:05:47.333953 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:05:47.333953 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:05:47.333953 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jun 29 11:05:47.333953 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:05:47.333953 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:05:47.333953 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:05:47.333953 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:05:47.340231 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:05:47.340231 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:05:47.340231 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:05:47.340231 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:05:47.340231 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 11:05:47.340231 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 11:05:47.340231 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 11:05:47.340231 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 11:05:47.340231 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 11:05:47.340231 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 11:05:47.340231 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 11:05:47.340231 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:05:47.340231 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 33/69] 10.222.0.44 () {56 vars in 1096 bytes} [Wed Jun 29 11:05:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:05:47.340231 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54064ac9-ed60-4b08-84a8-ec6905a807a9 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:05:47.358411 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6734195-4db7-4cd6-99cf-ec6bbecf4da7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:05:47.358411 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6734195-4db7-4cd6-99cf-ec6bbecf4da7 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:47.358411 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6734195-4db7-4cd6-99cf-ec6bbecf4da7 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:47.360134 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6734195-4db7-4cd6-99cf-ec6bbecf4da7 None admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:47.361627 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6734195-4db7-4cd6-99cf-ec6bbecf4da7 None admin] RBAC: Adding query filter params (name=member) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:05:47.362296 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6734195-4db7-4cd6-99cf-ec6bbecf4da7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:05:47.365413 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6734195-4db7-4cd6-99cf-ec6bbecf4da7 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:05:47.378108 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 37/70] 10.222.0.44 () {56 vars in 1101 bytes} [Wed Jun 29 11:05:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:05:47.381927 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a957ea5b-ea8d-4431-9038-73ff66512a2c None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:47.383077 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a957ea5b-ea8d-4431-9038-73ff66512a2c None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:47.383634 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a957ea5b-ea8d-4431-9038-73ff66512a2c None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:47.385171 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 38/71] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:05:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:05:47.386051 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a957ea5b-ea8d-4431-9038-73ff66512a2c None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:05:47.449632 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:05:47.457063 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:05:47.463650 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54064ac9-ed60-4b08-84a8-ec6905a807a9 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:05:47.500076 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a957ea5b-ea8d-4431-9038-73ff66512a2c None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:05:47.501311 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a70e2a4-8e34-4a74-9990-291243be57ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:05:47.502196 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a70e2a4-8e34-4a74-9990-291243be57ec None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:47.502489 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a70e2a4-8e34-4a74-9990-291243be57ec None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:47.502769 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a70e2a4-8e34-4a74-9990-291243be57ec None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:47.513897 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a70e2a4-8e34-4a74-9990-291243be57ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 11:05:47.522227 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-0a70e2a4-8e34-4a74-9990-291243be57ec None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 29 11:05:47.522227 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:05:47.522227 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:05:47.522227 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:05:47.522227 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:05:47.522227 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:05:47.522227 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:05:47.522227 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:05:47.522227 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:05:47.522227 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:05:47.522227 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:05:47.522227 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:05:47.522227 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 11:05:47.522227 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 11:05:47.522227 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 11:05:47.522227 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 11:05:47.523306 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 11:05:47.523306 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 11:05:47.523306 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 11:05:47.523306 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 11:05:47.523306 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:05:47.523306 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:05:47.523306 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:05:47.523306 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:05:47.523306 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:05:47.523306 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:05:47.523306 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:05:47.523306 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jun 29 11:05:47.523306 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:05:47.523306 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:05:47.523306 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:05:47.523306 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:05:47.524280 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:05:47.524280 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:05:47.524280 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:05:47.524280 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:05:47.524280 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 11:05:47.524280 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 11:05:47.524280 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 11:05:47.524280 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 11:05:47.524280 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 11:05:47.524280 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 11:05:47.524280 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 29 11:05:47.524280 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:05:47.524280 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 34/72] 10.222.0.44 () {56 vars in 1114 bytes} [Wed Jun 29 11:05:47 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:05:47.525059 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a70e2a4-8e34-4a74-9990-291243be57ec None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:05:47.543578 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4378834f-6cb6-4623-8c45-269fccf3dde3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:05:47.544343 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4378834f-6cb6-4623-8c45-269fccf3dde3 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:47.544581 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4378834f-6cb6-4623-8c45-269fccf3dde3 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:47.544866 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4378834f-6cb6-4623-8c45-269fccf3dde3 None admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:47.545626 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4378834f-6cb6-4623-8c45-269fccf3dde3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:05:47.545940 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4378834f-6cb6-4623-8c45-269fccf3dde3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:05:47.547338 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4378834f-6cb6-4623-8c45-269fccf3dde3 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:05:47.556051 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 39/73] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Jun 29 11:05:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:05:47.557237 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15362107-672e-47a1-911e-3151ac07096d None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:47.557531 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15362107-672e-47a1-911e-3151ac07096d None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:47.557812 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15362107-672e-47a1-911e-3151ac07096d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:47.574366 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-15362107-672e-47a1-911e-3151ac07096d None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:05:47.575507 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-15362107-672e-47a1-911e-3151ac07096d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:05:47.632438 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a70e2a4-8e34-4a74-9990-291243be57ec None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:05:47.643492 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 40/74] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:05:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 11:05:47.644853 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8522c8b6-26a5-4246-9cf3-7af47dc30441 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:47.645145 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8522c8b6-26a5-4246-9cf3-7af47dc30441 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:47.645428 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8522c8b6-26a5-4246-9cf3-7af47dc30441 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:47.646714 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 41/75] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:05:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:05:47.647400 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8522c8b6-26a5-4246-9cf3-7af47dc30441 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:05:47.669792 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e76ede2-56d1-43e2-99ea-897bf0a8c4d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:05:47.670665 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e76ede2-56d1-43e2-99ea-897bf0a8c4d4 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:47.670948 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e76ede2-56d1-43e2-99ea-897bf0a8c4d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:47.671221 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e76ede2-56d1-43e2-99ea-897bf0a8c4d4 None admin] PATH_INFO: `/v3/users/e2f3ce08c4a046fcae882fe888ca766d` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:47.686431 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e76ede2-56d1-43e2-99ea-897bf0a8c4d4 None admin] RBAC: Authorizing `identity:get_user(user_id=e2f3ce08c4a046fcae882fe888ca766d)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:05:47.687906 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e76ede2-56d1-43e2-99ea-897bf0a8c4d4 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:05:47.689539 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 35/76] 10.222.0.44 () {56 vars in 1177 bytes} [Wed Jun 29 11:05:47 2022] GET /identity/v3/users/e2f3ce08c4a046fcae882fe888ca766d => generated 275 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:05:47.690296 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e76ede2-56d1-43e2-99ea-897bf0a8c4d4 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:05:47.705615 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:05:47.707536 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:05:47.759282 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8522c8b6-26a5-4246-9cf3-7af47dc30441 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:05:47.787238 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e76ede2-56d1-43e2-99ea-897bf0a8c4d4 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:05:47.792289 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9d02ae-f544-4e5d-8df8-0100667bb5b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:05:47.792954 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9d02ae-f544-4e5d-8df8-0100667bb5b5 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:47.793242 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9d02ae-f544-4e5d-8df8-0100667bb5b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:47.793522 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9d02ae-f544-4e5d-8df8-0100667bb5b5 None admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:47.794273 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9d02ae-f544-4e5d-8df8-0100667bb5b5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:05:47.794583 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9d02ae-f544-4e5d-8df8-0100667bb5b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:05:47.796024 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9d02ae-f544-4e5d-8df8-0100667bb5b5 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:05:47.803847 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 42/77] 10.222.0.44 () {56 vars in 1111 bytes} [Wed Jun 29 11:05:47 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:05:47.805578 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b57fa0-5db3-4fd4-804b-cc3bcf9d800a None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:47.805870 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b57fa0-5db3-4fd4-804b-cc3bcf9d800a None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:47.806150 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b57fa0-5db3-4fd4-804b-cc3bcf9d800a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:47.821913 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f3d6f8-1883-4ad3-b6f5-fd69d152d693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:05:47.822085 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-f8b57fa0-5db3-4fd4-804b-cc3bcf9d800a None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:05:47.823052 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f3d6f8-1883-4ad3-b6f5-fd69d152d693 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:47.823341 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f3d6f8-1883-4ad3-b6f5-fd69d152d693 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:47.823478 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-f8b57fa0-5db3-4fd4-804b-cc3bcf9d800a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:05:47.823621 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f3d6f8-1883-4ad3-b6f5-fd69d152d693 None admin] PATH_INFO: `/v3/users/cdb4139035bd4d46a35dca750b630776` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:47.824915 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f3d6f8-1883-4ad3-b6f5-fd69d152d693 None admin] RBAC: Authorizing `identity:get_user(user_id=cdb4139035bd4d46a35dca750b630776)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:05:47.826304 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f3d6f8-1883-4ad3-b6f5-fd69d152d693 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:05:47.827957 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 36/78] 10.222.0.44 () {56 vars in 1177 bytes} [Wed Jun 29 11:05:47 2022] GET /identity/v3/users/cdb4139035bd4d46a35dca750b630776 => generated 247 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:05:47.828720 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f3d6f8-1883-4ad3-b6f5-fd69d152d693 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:05:47.887352 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 43/79] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:05:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 11:05:47.888217 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b57fa0-5db3-4fd4-804b-cc3bcf9d800a None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:05:47.914711 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:05:47.930908 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f3d6f8-1883-4ad3-b6f5-fd69d152d693 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:05:47.941128 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:05:47.964301 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb676a69-0280-4969-8fb2-0a587bd9e370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:05:47.965098 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb676a69-0280-4969-8fb2-0a587bd9e370 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:47.965388 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb676a69-0280-4969-8fb2-0a587bd9e370 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:47.965723 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb676a69-0280-4969-8fb2-0a587bd9e370 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:47.975786 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb676a69-0280-4969-8fb2-0a587bd9e370 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 11:05:47.984085 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-eb676a69-0280-4969-8fb2-0a587bd9e370 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 29 11:05:47.984085 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:05:47.984085 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:05:47.984085 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:05:47.984085 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:05:47.984085 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:05:47.984085 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:05:47.984085 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:05:47.984085 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:05:47.984085 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:05:47.984085 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:05:47.984085 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:05:47.984085 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 11:05:47.984085 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 11:05:47.984085 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 11:05:47.984085 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 11:05:47.985049 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 11:05:47.985049 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 11:05:47.985049 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 11:05:47.985049 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 11:05:47.985049 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:05:47.985049 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:05:47.985049 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:05:47.985049 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:05:47.985049 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:05:47.985049 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:05:47.985049 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:05:47.985049 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jun 29 11:05:47.985049 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:05:47.985049 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:05:47.985049 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:05:47.985049 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:05:47.986026 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:05:47.986026 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:05:47.986026 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:05:47.986026 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:05:47.986026 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 11:05:47.986026 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 11:05:47.986026 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 11:05:47.986026 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 11:05:47.986026 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 11:05:47.986026 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 11:05:47.986026 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 29 11:05:47.986026 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:05:47.986026 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 37/80] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Jun 29 11:05:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:05:47.986026 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb676a69-0280-4969-8fb2-0a587bd9e370 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:05:48.003015 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b57fa0-5db3-4fd4-804b-cc3bcf9d800a None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:05:48.037792 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df10343-67e8-40dd-9ce6-bb4d0593c449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:05:48.038443 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df10343-67e8-40dd-9ce6-bb4d0593c449 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:48.038751 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df10343-67e8-40dd-9ce6-bb4d0593c449 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:48.039014 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df10343-67e8-40dd-9ce6-bb4d0593c449 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:48.083910 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb676a69-0280-4969-8fb2-0a587bd9e370 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:05:48.104650 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df10343-67e8-40dd-9ce6-bb4d0593c449 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:05:48.106123 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df10343-67e8-40dd-9ce6-bb4d0593c449 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:05:48.116456 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc4c681-c26d-4fc1-b0d9-edaea4941d9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:05:48.117363 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc4c681-c26d-4fc1-b0d9-edaea4941d9b None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:48.117654 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc4c681-c26d-4fc1-b0d9-edaea4941d9b None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:48.117997 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc4c681-c26d-4fc1-b0d9-edaea4941d9b None admin] PATH_INFO: `/v3/users/e2f3ce08c4a046fcae882fe888ca766d` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:48.119227 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc4c681-c26d-4fc1-b0d9-edaea4941d9b None admin] RBAC: Authorizing `identity:get_user(user_id=e2f3ce08c4a046fcae882fe888ca766d)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:05:48.120707 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc4c681-c26d-4fc1-b0d9-edaea4941d9b None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:05:48.122334 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 38/81] 10.222.0.44 () {56 vars in 1177 bytes} [Wed Jun 29 11:05:47 2022] GET /identity/v3/users/e2f3ce08c4a046fcae882fe888ca766d => generated 275 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:05:48.127485 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc4c681-c26d-4fc1-b0d9-edaea4941d9b None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:05:48.139931 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:05:48.143720 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:05:48.202285 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 44/82] 10.222.0.44 () {58 vars in 1282 bytes} [Wed Jun 29 11:05:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 29 11:05:48.203168 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df10343-67e8-40dd-9ce6-bb4d0593c449 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:05:48.226244 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc4c681-c26d-4fc1-b0d9-edaea4941d9b None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:05:48.260143 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101f1076-213c-4b66-bda7-bcf979b06553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:05:48.260796 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101f1076-213c-4b66-bda7-bcf979b06553 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:48.261086 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101f1076-213c-4b66-bda7-bcf979b06553 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:48.261372 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101f1076-213c-4b66-bda7-bcf979b06553 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:48.300981 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df10343-67e8-40dd-9ce6-bb4d0593c449 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:05:48.328776 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101f1076-213c-4b66-bda7-bcf979b06553 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:05:48.330250 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101f1076-213c-4b66-bda7-bcf979b06553 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:05:48.334638 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4fdff8-0910-4783-80a8-c031fb8e5c61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:05:48.335376 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4fdff8-0910-4783-80a8-c031fb8e5c61 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:48.335692 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4fdff8-0910-4783-80a8-c031fb8e5c61 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:48.336009 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4fdff8-0910-4783-80a8-c031fb8e5c61 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:48.337195 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4fdff8-0910-4783-80a8-c031fb8e5c61 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:05:48.338647 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4fdff8-0910-4783-80a8-c031fb8e5c61 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:05:48.340323 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 45/83] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jun 29 11:05:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:05:48.341092 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4fdff8-0910-4783-80a8-c031fb8e5c61 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:05:48.345261 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:05:48.352579 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:05:48.428723 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 39/84] 10.222.0.44 () {58 vars in 1282 bytes} [Wed Jun 29 11:05:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 29 11:05:48.430162 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101f1076-213c-4b66-bda7-bcf979b06553 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:05:48.444321 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4fdff8-0910-4783-80a8-c031fb8e5c61 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:05:48.479358 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5e4ff6-10e9-4e80-95df-2edd21068b7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:05:48.480059 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5e4ff6-10e9-4e80-95df-2edd21068b7f None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:48.480348 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5e4ff6-10e9-4e80-95df-2edd21068b7f None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:48.480996 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5e4ff6-10e9-4e80-95df-2edd21068b7f None admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:48.481761 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5e4ff6-10e9-4e80-95df-2edd21068b7f None admin] RBAC: Adding query filter params (name=member) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:05:48.482214 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5e4ff6-10e9-4e80-95df-2edd21068b7f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:05:48.486054 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5e4ff6-10e9-4e80-95df-2edd21068b7f None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:05:48.496585 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 46/85] 10.222.0.44 () {56 vars in 1101 bytes} [Wed Jun 29 11:05:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:05:48.497398 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5e4ff6-10e9-4e80-95df-2edd21068b7f None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:05:48.534913 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101f1076-213c-4b66-bda7-bcf979b06553 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:05:48.544946 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:05:48.562412 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:05:48.567864 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78f7d22-8ce6-4dc9-a3bb-3d13f0806c45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:05:48.568518 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78f7d22-8ce6-4dc9-a3bb-3d13f0806c45 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:48.568809 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78f7d22-8ce6-4dc9-a3bb-3d13f0806c45 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:48.569091 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78f7d22-8ce6-4dc9-a3bb-3d13f0806c45 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:48.594691 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5e4ff6-10e9-4e80-95df-2edd21068b7f None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:05:48.627447 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9332de-62f2-4df9-ba69-77965673a1c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:05:48.628328 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9332de-62f2-4df9-ba69-77965673a1c8 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:48.628609 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9332de-62f2-4df9-ba69-77965673a1c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:48.628883 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9332de-62f2-4df9-ba69-77965673a1c8 None admin] PATH_INFO: `/v3/projects/3b98d22914de4e3b8bca7fe0a0b51357` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:48.630060 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9332de-62f2-4df9-ba69-77965673a1c8 None admin] RBAC: Authorizing `identity:get_project(project_id=3b98d22914de4e3b8bca7fe0a0b51357)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:05:48.631505 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9332de-62f2-4df9-ba69-77965673a1c8 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:05:48.637947 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 47/86] 10.222.0.44 () {56 vars in 1186 bytes} [Wed Jun 29 11:05:48 2022] GET /identity/v3/projects/3b98d22914de4e3b8bca7fe0a0b51357 => generated 298 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:05:48.637947 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9332de-62f2-4df9-ba69-77965673a1c8 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:05:48.638131 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78f7d22-8ce6-4dc9-a3bb-3d13f0806c45 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:05:48.638955 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78f7d22-8ce6-4dc9-a3bb-3d13f0806c45 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:05:48.731754 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9332de-62f2-4df9-ba69-77965673a1c8 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:05:48.736162 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 40/87] 10.222.0.44 () {58 vars in 1282 bytes} [Wed Jun 29 11:05:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 29 11:05:48.736993 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78f7d22-8ce6-4dc9-a3bb-3d13f0806c45 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:05:48.749900 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:05:48.765208 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87be416-c589-4d6c-a5bd-3dd9668ec4f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:05:48.765880 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87be416-c589-4d6c-a5bd-3dd9668ec4f9 None admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:48.768665 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87be416-c589-4d6c-a5bd-3dd9668ec4f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:48.768665 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87be416-c589-4d6c-a5bd-3dd9668ec4f9 None admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:48.768665 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87be416-c589-4d6c-a5bd-3dd9668ec4f9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:05:48.768665 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87be416-c589-4d6c-a5bd-3dd9668ec4f9 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:05:48.776512 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:05:48.834587 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78f7d22-8ce6-4dc9-a3bb-3d13f0806c45 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:05:48.867588 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ab57a5-c392-4480-a661-16c771e587c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:05:48.868417 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ab57a5-c392-4480-a661-16c771e587c7 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:48.868712 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ab57a5-c392-4480-a661-16c771e587c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:48.868990 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ab57a5-c392-4480-a661-16c771e587c7 None admin] PATH_INFO: `/v3/projects/3b98d22914de4e3b8bca7fe0a0b51357` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:48.870185 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ab57a5-c392-4480-a661-16c771e587c7 None admin] RBAC: Authorizing `identity:get_project(project_id=3b98d22914de4e3b8bca7fe0a0b51357)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:05:48.871635 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ab57a5-c392-4480-a661-16c771e587c7 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:05:48.873374 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 41/88] 10.222.0.44 () {56 vars in 1186 bytes} [Wed Jun 29 11:05:48 2022] GET /identity/v3/projects/3b98d22914de4e3b8bca7fe0a0b51357 => generated 298 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:05:48.880187 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 48/89] 10.222.0.44 () {60 vars in 1132 bytes} [Wed Jun 29 11:05:48 2022] POST /identity/v3/users => generated 289 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:05:48.880697 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ab57a5-c392-4480-a661-16c771e587c7 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:05:48.880799 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87be416-c589-4d6c-a5bd-3dd9668ec4f9 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:05:48.973925 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:05:48.993890 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87be416-c589-4d6c-a5bd-3dd9668ec4f9 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:05:48.996905 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ab57a5-c392-4480-a661-16c771e587c7 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:05:49.033452 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a0a613-0316-4a52-bb81-0dcde37818d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:05:49.034123 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a0a613-0316-4a52-bb81-0dcde37818d4 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:49.034410 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a0a613-0316-4a52-bb81-0dcde37818d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:49.034690 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a0a613-0316-4a52-bb81-0dcde37818d4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:49.036259 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a0a613-0316-4a52-bb81-0dcde37818d4 None admin] RBAC: Adding query filter params (role.id=e1c88eebe911432cab6a07fbeb84653b, scope.project.id=3b98d22914de4e3b8bca7fe0a0b51357, user.id=e2f3ce08c4a046fcae882fe888ca766d) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:05:49.036259 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a0a613-0316-4a52-bb81-0dcde37818d4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:05:49.036434 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3644c89-9d9e-43e3-b218-3f4e4e8d77fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:05:49.036950 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3644c89-9d9e-43e3-b218-3f4e4e8d77fe None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:49.037498 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3644c89-9d9e-43e3-b218-3f4e4e8d77fe None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:49.037639 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a0a613-0316-4a52-bb81-0dcde37818d4 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:05:49.037786 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3644c89-9d9e-43e3-b218-3f4e4e8d77fe None admin] PATH_INFO: `/v3/users/e2f3ce08c4a046fcae882fe888ca766d` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:49.039305 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3644c89-9d9e-43e3-b218-3f4e4e8d77fe None admin] RBAC: Authorizing `identity:get_user(user_id=e2f3ce08c4a046fcae882fe888ca766d)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:05:49.040814 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3644c89-9d9e-43e3-b218-3f4e4e8d77fe None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:05:49.042636 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 42/90] 10.222.0.44 () {56 vars in 1177 bytes} [Wed Jun 29 11:05:48 2022] GET /identity/v3/users/e2f3ce08c4a046fcae882fe888ca766d => generated 275 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:05:49.043311 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3644c89-9d9e-43e3-b218-3f4e4e8d77fe None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:05:49.045330 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 49/91] 10.222.0.44 () {56 vars in 1374 bytes} [Wed Jun 29 11:05:48 2022] GET /identity/v3/role_assignments?user.id=e2f3ce08c4a046fcae882fe888ca766d&scope.project.id=3b98d22914de4e3b8bca7fe0a0b51357&role.id=e1c88eebe911432cab6a07fbeb84653b => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:05:49.047570 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a0a613-0316-4a52-bb81-0dcde37818d4 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:05:49.133257 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:05:49.191149 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3644c89-9d9e-43e3-b218-3f4e4e8d77fe None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:05:49.197545 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a0a613-0316-4a52-bb81-0dcde37818d4 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:05:49.241641 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568df4ed-e1b7-4662-bb16-ec268c544fe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:05:49.243317 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568df4ed-e1b7-4662-bb16-ec268c544fe6 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:49.243753 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568df4ed-e1b7-4662-bb16-ec268c544fe6 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:49.244218 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568df4ed-e1b7-4662-bb16-ec268c544fe6 None admin] PATH_INFO: `/v3/projects/3b98d22914de4e3b8bca7fe0a0b51357` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:49.245630 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568df4ed-e1b7-4662-bb16-ec268c544fe6 None admin] RBAC: Authorizing `identity:get_project(project_id=3b98d22914de4e3b8bca7fe0a0b51357)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:05:49.247295 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568df4ed-e1b7-4662-bb16-ec268c544fe6 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:05:49.247664 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78ab94f-4e39-4464-993b-4b39fcc78717 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:05:49.248366 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78ab94f-4e39-4464-993b-4b39fcc78717 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:49.248654 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78ab94f-4e39-4464-993b-4b39fcc78717 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:49.248942 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78ab94f-4e39-4464-993b-4b39fcc78717 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:49.249806 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78ab94f-4e39-4464-993b-4b39fcc78717 None admin] RBAC: Adding query filter params (role.id=e95c2ce63e114cc19a9dda6ad21e794f, scope.project.id=3b98d22914de4e3b8bca7fe0a0b51357, user.id=cdb4139035bd4d46a35dca750b630776) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:05:49.250032 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 43/92] 10.222.0.44 () {56 vars in 1186 bytes} [Wed Jun 29 11:05:49 2022] GET /identity/v3/projects/3b98d22914de4e3b8bca7fe0a0b51357 => generated 298 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:05:49.250134 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78ab94f-4e39-4464-993b-4b39fcc78717 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:05:49.251032 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568df4ed-e1b7-4662-bb16-ec268c544fe6 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:05:49.251554 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78ab94f-4e39-4464-993b-4b39fcc78717 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:05:49.259476 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 50/93] 10.222.0.44 () {56 vars in 1374 bytes} [Wed Jun 29 11:05:49 2022] GET /identity/v3/role_assignments?user.id=cdb4139035bd4d46a35dca750b630776&scope.project.id=3b98d22914de4e3b8bca7fe0a0b51357&role.id=e95c2ce63e114cc19a9dda6ad21e794f => generated 259 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:05:49.260389 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78ab94f-4e39-4464-993b-4b39fcc78717 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:05:49.281171 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:05:49.394871 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78ab94f-4e39-4464-993b-4b39fcc78717 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:05:49.428392 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:05:49.450810 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6933e08d-ff59-488a-a547-38af043e0ec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:05:49.451497 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6933e08d-ff59-488a-a547-38af043e0ec0 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:49.451788 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6933e08d-ff59-488a-a547-38af043e0ec0 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:49.452121 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6933e08d-ff59-488a-a547-38af043e0ec0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:49.453411 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6933e08d-ff59-488a-a547-38af043e0ec0 None admin] RBAC: Adding query filter params (role.id=fc91ce4ad4a44cc3b05d0936754e87ec, scope.project.id=3b98d22914de4e3b8bca7fe0a0b51357, user.id=e2f3ce08c4a046fcae882fe888ca766d) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:05:49.453732 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6933e08d-ff59-488a-a547-38af043e0ec0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:05:49.455387 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6933e08d-ff59-488a-a547-38af043e0ec0 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:05:49.462877 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 51/94] 10.222.0.44 () {56 vars in 1374 bytes} [Wed Jun 29 11:05:49 2022] GET /identity/v3/role_assignments?user.id=e2f3ce08c4a046fcae882fe888ca766d&scope.project.id=3b98d22914de4e3b8bca7fe0a0b51357&role.id=fc91ce4ad4a44cc3b05d0936754e87ec => generated 259 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:05:49.491394 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568df4ed-e1b7-4662-bb16-ec268c544fe6 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:05:49.534196 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f33cacb-2c77-4f78-a901-82c4e1d371b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:05:49.535130 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f33cacb-2c77-4f78-a901-82c4e1d371b9 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:49.535562 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f33cacb-2c77-4f78-a901-82c4e1d371b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:49.536094 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f33cacb-2c77-4f78-a901-82c4e1d371b9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:49.618867 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f33cacb-2c77-4f78-a901-82c4e1d371b9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:05:49.620637 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f33cacb-2c77-4f78-a901-82c4e1d371b9 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:05:49.647622 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:05:49.759040 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 44/95] 10.222.0.44 () {58 vars in 1282 bytes} [Wed Jun 29 11:05:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 508 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 29 11:05:49.765631 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6933e08d-ff59-488a-a547-38af043e0ec0 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:05:49.930150 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6933e08d-ff59-488a-a547-38af043e0ec0 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:05:49.984628 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb5cbe3-0014-43f9-9dcb-fe1f10513e6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:05:49.985515 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb5cbe3-0014-43f9-9dcb-fe1f10513e6f None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:49.985802 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb5cbe3-0014-43f9-9dcb-fe1f10513e6f None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:49.986111 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb5cbe3-0014-43f9-9dcb-fe1f10513e6f None admin] PATH_INFO: `/v3/projects/89118c554be440f7b5287f08494d31b6` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:49.988613 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb5cbe3-0014-43f9-9dcb-fe1f10513e6f None admin] RBAC: Authorizing `identity:get_project(project_id=89118c554be440f7b5287f08494d31b6)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:05:49.990100 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb5cbe3-0014-43f9-9dcb-fe1f10513e6f None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:05:49.992584 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 52/96] 10.222.0.44 () {56 vars in 1186 bytes} [Wed Jun 29 11:05:49 2022] GET /identity/v3/projects/89118c554be440f7b5287f08494d31b6 => generated 312 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:05:49.998346 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f33cacb-2c77-4f78-a901-82c4e1d371b9 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:05:50.165163 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f33cacb-2c77-4f78-a901-82c4e1d371b9 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:05:50.186695 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:05:50.212803 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481c454c-2bdc-4548-ac57-c5c17ed4c99c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:05:50.213812 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481c454c-2bdc-4548-ac57-c5c17ed4c99c None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:50.214293 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481c454c-2bdc-4548-ac57-c5c17ed4c99c None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:50.214783 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481c454c-2bdc-4548-ac57-c5c17ed4c99c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:50.215820 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481c454c-2bdc-4548-ac57-c5c17ed4c99c None admin] RBAC: Adding query filter params (role.id=e1c88eebe911432cab6a07fbeb84653b, scope.project.id=89118c554be440f7b5287f08494d31b6, user.id=e2f3ce08c4a046fcae882fe888ca766d) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:05:50.216420 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481c454c-2bdc-4548-ac57-c5c17ed4c99c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:05:50.218071 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481c454c-2bdc-4548-ac57-c5c17ed4c99c None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:05:50.230668 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 45/97] 10.222.0.44 () {56 vars in 1374 bytes} [Wed Jun 29 11:05:49 2022] GET /identity/v3/role_assignments?user.id=e2f3ce08c4a046fcae882fe888ca766d&scope.project.id=89118c554be440f7b5287f08494d31b6&role.id=e1c88eebe911432cab6a07fbeb84653b => generated 259 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:05:51.865556 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d44aa9-2c5f-4384-8578-ee4a1a7fe585 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:51.866062 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d44aa9-2c5f-4384-8578-ee4a1a7fe585 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:51.866135 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d44aa9-2c5f-4384-8578-ee4a1a7fe585 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:51.868186 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 53/98] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:05:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:05:51.873627 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae1971f-10c6-472a-b226-14d37b10be64 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:51.874015 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae1971f-10c6-472a-b226-14d37b10be64 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:51.874193 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae1971f-10c6-472a-b226-14d37b10be64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:51.900794 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-0ae1971f-10c6-472a-b226-14d37b10be64 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:05:51.902313 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-0ae1971f-10c6-472a-b226-14d37b10be64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:05:51.988063 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 46/99] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:05:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 11:05:51.990726 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04860d70-bafb-4cfa-baf1-d17d8c2e9e7a None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:51.991017 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04860d70-bafb-4cfa-baf1-d17d8c2e9e7a None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:51.991298 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04860d70-bafb-4cfa-baf1-d17d8c2e9e7a None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:51.992714 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 54/100] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:05:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:05:51.997802 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4dafe4-2a07-455e-b7c5-a8f57a45401a None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:51.998098 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4dafe4-2a07-455e-b7c5-a8f57a45401a None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:51.998379 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4dafe4-2a07-455e-b7c5-a8f57a45401a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:52.024765 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-8e4dafe4-2a07-455e-b7c5-a8f57a45401a None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:05:52.025947 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-8e4dafe4-2a07-455e-b7c5-a8f57a45401a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:05:52.030005 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04860d70-bafb-4cfa-baf1-d17d8c2e9e7a None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:05:52.140451 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 47/101] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:05:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 11:05:52.141111 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:05:52.169897 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4dafe4-2a07-455e-b7c5-a8f57a45401a None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:05:52.232617 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04860d70-bafb-4cfa-baf1-d17d8c2e9e7a None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:05:52.244840 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:05:52.312597 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fc8840-0e74-45f4-a969-841f06f9b3bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:05:52.313652 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fc8840-0e74-45f4-a969-841f06f9b3bb None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:52.314080 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fc8840-0e74-45f4-a969-841f06f9b3bb None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:52.314526 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fc8840-0e74-45f4-a969-841f06f9b3bb None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:52.342771 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fc8840-0e74-45f4-a969-841f06f9b3bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 11:05:52.351172 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4dafe4-2a07-455e-b7c5-a8f57a45401a None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:05:52.358059 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-84fc8840-0e74-45f4-a969-841f06f9b3bb None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 29 11:05:52.358059 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:05:52.358059 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:05:52.358059 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:05:52.358059 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:05:52.358059 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:05:52.358059 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:05:52.358059 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:05:52.358059 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:05:52.358059 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:05:52.358059 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:05:52.358059 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:05:52.358059 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 11:05:52.358059 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 11:05:52.358059 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 11:05:52.358059 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 11:05:52.364252 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 11:05:52.364252 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 11:05:52.364252 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 11:05:52.364252 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 11:05:52.364252 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:05:52.364252 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:05:52.364252 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:05:52.364252 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:05:52.364252 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:05:52.364252 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:05:52.364252 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:05:52.364252 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jun 29 11:05:52.364252 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:05:52.364252 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:05:52.364252 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:05:52.364252 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:05:52.365028 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:05:52.365028 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:05:52.365028 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:05:52.365028 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:05:52.365028 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 11:05:52.365028 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 11:05:52.365028 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 11:05:52.365028 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 11:05:52.365028 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 11:05:52.365028 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 11:05:52.365028 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 29 11:05:52.365028 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jun 29 11:05:52.367127 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 55/102] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Jun 29 11:05:52 2022] GET /identity/v3/roles/reader => generated 84 bytes in 338 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 11:05:52.373404 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fc8840-0e74-45f4-a969-841f06f9b3bb None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:05:52.417009 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0f2c5c-635b-44da-bac5-8d8ec90cdba7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:05:52.417720 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0f2c5c-635b-44da-bac5-8d8ec90cdba7 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:52.418005 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0f2c5c-635b-44da-bac5-8d8ec90cdba7 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:52.418283 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0f2c5c-635b-44da-bac5-8d8ec90cdba7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:52.428583 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0f2c5c-635b-44da-bac5-8d8ec90cdba7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 11:05:52.436618 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-ac0f2c5c-635b-44da-bac5-8d8ec90cdba7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 29 11:05:52.436618 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:05:52.436618 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:05:52.436618 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:05:52.436618 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:05:52.436618 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:05:52.436618 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:05:52.436618 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:05:52.436618 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:05:52.436618 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:05:52.436618 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:05:52.436618 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:05:52.436618 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 11:05:52.436618 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 11:05:52.436618 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 11:05:52.436618 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 11:05:52.439452 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 11:05:52.439452 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 11:05:52.439452 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 11:05:52.439452 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 11:05:52.439452 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:05:52.439452 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:05:52.439452 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:05:52.439452 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:05:52.439452 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:05:52.439452 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:05:52.439452 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:05:52.439452 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jun 29 11:05:52.439452 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:05:52.439452 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:05:52.439452 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:05:52.439452 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:05:52.440448 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:05:52.440448 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:05:52.440448 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:05:52.440448 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:05:52.440448 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 11:05:52.440448 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 11:05:52.440448 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 11:05:52.440448 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 11:05:52.440448 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 11:05:52.440448 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 11:05:52.440448 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 29 11:05:52.440448 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:05:52.440448 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 48/103] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Jun 29 11:05:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 11:05:52.440448 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655c635f-bc72-4f73-a1f6-5ed5d334801b None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:52.441315 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655c635f-bc72-4f73-a1f6-5ed5d334801b None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:52.441315 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655c635f-bc72-4f73-a1f6-5ed5d334801b None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:52.441315 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 49/104] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:05:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:05:52.448607 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8671e94c-8b8c-4c37-b0e2-4646fc6f050e None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:52.448980 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8671e94c-8b8c-4c37-b0e2-4646fc6f050e None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:52.449324 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8671e94c-8b8c-4c37-b0e2-4646fc6f050e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:52.467777 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-8671e94c-8b8c-4c37-b0e2-4646fc6f050e None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:05:52.470577 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-8671e94c-8b8c-4c37-b0e2-4646fc6f050e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:05:52.554271 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 50/105] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:05:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 29 11:05:52.555872 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8671e94c-8b8c-4c37-b0e2-4646fc6f050e None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:05:52.576938 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fc8840-0e74-45f4-a969-841f06f9b3bb None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:05:52.583230 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:05:52.589964 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:05:52.611158 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac86195-f2d0-4818-af42-2abd8d49cd6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:05:52.611848 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac86195-f2d0-4818-af42-2abd8d49cd6d None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:52.612161 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac86195-f2d0-4818-af42-2abd8d49cd6d None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:52.612440 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac86195-f2d0-4818-af42-2abd8d49cd6d None admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:52.613193 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac86195-f2d0-4818-af42-2abd8d49cd6d None admin] RBAC: Adding query filter params (name=reader) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:05:52.613504 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac86195-f2d0-4818-af42-2abd8d49cd6d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:05:52.614985 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac86195-f2d0-4818-af42-2abd8d49cd6d None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:05:52.624214 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 56/106] 10.222.0.44 () {56 vars in 1101 bytes} [Wed Jun 29 11:05:52 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 11:05:52.625437 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727b46d7-ae3f-4d7e-9325-3a9a3c1abf4f None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:52.625729 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727b46d7-ae3f-4d7e-9325-3a9a3c1abf4f None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:52.626010 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727b46d7-ae3f-4d7e-9325-3a9a3c1abf4f None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:52.631579 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 57/107] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:05:52 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:05:52.631579 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff954b62-3ab7-4708-af57-26802a0dd68c None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:52.631579 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff954b62-3ab7-4708-af57-26802a0dd68c None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:52.631579 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff954b62-3ab7-4708-af57-26802a0dd68c None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:52.631579 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 58/108] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:05:52 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:05:52.631579 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff954b62-3ab7-4708-af57-26802a0dd68c None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:05:52.668927 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8671e94c-8b8c-4c37-b0e2-4646fc6f050e None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:05:52.705230 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86780a54-ad4c-4e58-ac8f-fcda81351cf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:05:52.705892 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86780a54-ad4c-4e58-ac8f-fcda81351cf1 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:52.706181 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86780a54-ad4c-4e58-ac8f-fcda81351cf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:52.706456 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86780a54-ad4c-4e58-ac8f-fcda81351cf1 None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:52.707202 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86780a54-ad4c-4e58-ac8f-fcda81351cf1 None admin] RBAC: Adding query filter params (name=member) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:05:52.707504 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86780a54-ad4c-4e58-ac8f-fcda81351cf1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:05:52.708927 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86780a54-ad4c-4e58-ac8f-fcda81351cf1 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:05:52.717679 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 51/109] 10.222.0.44 () {56 vars in 1101 bytes} [Wed Jun 29 11:05:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:05:52.722927 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86780a54-ad4c-4e58-ac8f-fcda81351cf1 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:05:52.738953 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff954b62-3ab7-4708-af57-26802a0dd68c None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:05:52.780043 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a4dca1-cdf6-4eee-9ee0-86d1cd4c50dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:05:52.781124 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a4dca1-cdf6-4eee-9ee0-86d1cd4c50dd None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:52.781602 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a4dca1-cdf6-4eee-9ee0-86d1cd4c50dd None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:52.781849 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a4dca1-cdf6-4eee-9ee0-86d1cd4c50dd None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:52.792996 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a4dca1-cdf6-4eee-9ee0-86d1cd4c50dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 11:05:52.801148 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-23a4dca1-cdf6-4eee-9ee0-86d1cd4c50dd None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 29 11:05:52.801148 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:05:52.801148 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:05:52.801148 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:05:52.801148 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:05:52.801148 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:05:52.801148 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:05:52.801148 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:05:52.801148 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:05:52.801148 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:05:52.801148 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:05:52.801148 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:05:52.801148 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 11:05:52.801148 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 11:05:52.801148 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 11:05:52.801148 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 11:05:52.802396 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 11:05:52.802396 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 11:05:52.802396 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 11:05:52.802396 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 11:05:52.802396 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:05:52.802396 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:05:52.802396 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:05:52.802396 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:05:52.802396 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:05:52.802396 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:05:52.802396 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:05:52.802396 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jun 29 11:05:52.802396 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:05:52.802396 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:05:52.802396 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:05:52.802396 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:05:52.803184 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:05:52.803184 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:05:52.803184 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:05:52.803184 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:05:52.803184 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 11:05:52.803184 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 11:05:52.803184 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 11:05:52.803184 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 11:05:52.803184 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 11:05:52.803184 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 11:05:52.803184 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 29 11:05:52.803184 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jun 29 11:05:52.803184 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 59/110] 10.222.0.44 () {56 vars in 1114 bytes} [Wed Jun 29 11:05:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:05:52.803821 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67ac80b-3c8a-490b-9123-1d9ba5ef1994 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:52.803821 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67ac80b-3c8a-490b-9123-1d9ba5ef1994 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:52.805211 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67ac80b-3c8a-490b-9123-1d9ba5ef1994 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:52.822728 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-b67ac80b-3c8a-490b-9123-1d9ba5ef1994 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:05:52.823981 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-b67ac80b-3c8a-490b-9123-1d9ba5ef1994 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:05:52.834296 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:05:52.843525 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86780a54-ad4c-4e58-ac8f-fcda81351cf1 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:05:52.851272 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:05:52.877658 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcdb85b-2e62-4cbc-b02b-bcdadfc0b25a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:05:52.878580 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcdb85b-2e62-4cbc-b02b-bcdadfc0b25a None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:52.879133 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcdb85b-2e62-4cbc-b02b-bcdadfc0b25a None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:52.879439 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcdb85b-2e62-4cbc-b02b-bcdadfc0b25a None admin] PATH_INFO: `/v3/users/c444598ebd8845d2a9cdf41cefa70e01` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:52.894576 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 60/111] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:05:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 11:05:52.895778 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0e18e8-25e8-4ca5-a515-58015ae93072 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:52.896077 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0e18e8-25e8-4ca5-a515-58015ae93072 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:52.896723 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0e18e8-25e8-4ca5-a515-58015ae93072 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:52.904143 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcdb85b-2e62-4cbc-b02b-bcdadfc0b25a None admin] RBAC: Authorizing `identity:get_user(user_id=c444598ebd8845d2a9cdf41cefa70e01)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:05:52.905900 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcdb85b-2e62-4cbc-b02b-bcdadfc0b25a None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:05:52.913608 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 52/112] 10.222.0.44 () {56 vars in 1177 bytes} [Wed Jun 29 11:05:52 2022] GET /identity/v3/users/c444598ebd8845d2a9cdf41cefa70e01 => generated 289 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:05:52.914666 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcdb85b-2e62-4cbc-b02b-bcdadfc0b25a None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:05:52.921323 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-fc0e18e8-25e8-4ca5-a515-58015ae93072 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:05:52.922529 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-fc0e18e8-25e8-4ca5-a515-58015ae93072 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:05:52.992894 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 61/113] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:05:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 11:05:52.993757 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0e18e8-25e8-4ca5-a515-58015ae93072 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:05:53.030572 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcdb85b-2e62-4cbc-b02b-bcdadfc0b25a None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:05:53.066057 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd8b103-66b2-443c-9af4-5561c81e80bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:05:53.067228 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd8b103-66b2-443c-9af4-5561c81e80bf None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:53.067509 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd8b103-66b2-443c-9af4-5561c81e80bf None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:53.067800 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd8b103-66b2-443c-9af4-5561c81e80bf None admin] PATH_INFO: `/v3/users/e2f3ce08c4a046fcae882fe888ca766d` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:53.069038 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd8b103-66b2-443c-9af4-5561c81e80bf None admin] RBAC: Authorizing `identity:get_user(user_id=e2f3ce08c4a046fcae882fe888ca766d)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:05:53.071053 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd8b103-66b2-443c-9af4-5561c81e80bf None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:05:53.077413 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 53/114] 10.222.0.44 () {56 vars in 1177 bytes} [Wed Jun 29 11:05:52 2022] GET /identity/v3/users/e2f3ce08c4a046fcae882fe888ca766d => generated 275 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:05:53.077413 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd8b103-66b2-443c-9af4-5561c81e80bf None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:05:53.094236 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:05:53.107137 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:05:53.113806 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0e18e8-25e8-4ca5-a515-58015ae93072 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:05:53.147883 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6ead85-40c7-41de-b0e6-785fba6f3e8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:05:53.148570 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6ead85-40c7-41de-b0e6-785fba6f3e8c None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:53.148858 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6ead85-40c7-41de-b0e6-785fba6f3e8c None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:53.149139 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6ead85-40c7-41de-b0e6-785fba6f3e8c None admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:53.149897 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6ead85-40c7-41de-b0e6-785fba6f3e8c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:05:53.150208 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6ead85-40c7-41de-b0e6-785fba6f3e8c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:05:53.151605 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6ead85-40c7-41de-b0e6-785fba6f3e8c None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:05:53.165035 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 62/115] 10.222.0.44 () {56 vars in 1111 bytes} [Wed Jun 29 11:05:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:05:53.165878 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6ead85-40c7-41de-b0e6-785fba6f3e8c None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:05:53.183351 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd8b103-66b2-443c-9af4-5561c81e80bf None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:05:53.217901 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ff8fe1-45e6-4135-a34e-fedf529692cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:05:53.218554 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ff8fe1-45e6-4135-a34e-fedf529692cb None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:53.218844 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ff8fe1-45e6-4135-a34e-fedf529692cb None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:53.219119 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ff8fe1-45e6-4135-a34e-fedf529692cb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:53.276147 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6ead85-40c7-41de-b0e6-785fba6f3e8c None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:05:53.285632 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ff8fe1-45e6-4135-a34e-fedf529692cb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:05:53.287110 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ff8fe1-45e6-4135-a34e-fedf529692cb None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:05:53.303952 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:05:53.318840 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45abe29f-3548-48b4-b95d-06241e9a70fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:05:53.319657 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45abe29f-3548-48b4-b95d-06241e9a70fa None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:53.319976 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45abe29f-3548-48b4-b95d-06241e9a70fa None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:53.320310 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45abe29f-3548-48b4-b95d-06241e9a70fa None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:53.330842 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45abe29f-3548-48b4-b95d-06241e9a70fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 11:05:53.338602 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-45abe29f-3548-48b4-b95d-06241e9a70fa None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 11:05:53.338602 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:05:53.338602 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:05:53.338602 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:05:53.338602 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:05:53.338602 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:05:53.338602 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:05:53.338602 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:05:53.338602 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:05:53.338602 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:05:53.338602 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:05:53.338602 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:05:53.338602 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 11:05:53.338602 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 11:05:53.338602 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 11:05:53.338602 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 11:05:53.339473 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 11:05:53.339473 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 11:05:53.339473 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 11:05:53.339473 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 11:05:53.339473 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:05:53.339473 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:05:53.339473 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:05:53.339473 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:05:53.339473 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:05:53.339473 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:05:53.339473 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:05:53.339473 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jun 29 11:05:53.339473 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:05:53.339473 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:05:53.339473 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:05:53.339473 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:05:53.340267 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:05:53.340267 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:05:53.340267 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:05:53.340267 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:05:53.340267 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 11:05:53.340267 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 11:05:53.340267 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 11:05:53.340267 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 11:05:53.340267 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 11:05:53.340267 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 11:05:53.340267 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 11:05:53.340267 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jun 29 11:05:53.340267 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 63/116] 10.222.0.44 () {56 vars in 1096 bytes} [Wed Jun 29 11:05:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:05:53.340898 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45abe29f-3548-48b4-b95d-06241e9a70fa None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:05:53.360022 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:05:53.391510 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 54/117] 10.222.0.44 () {58 vars in 1282 bytes} [Wed Jun 29 11:05:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 29 11:05:53.392484 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ff8fe1-45e6-4135-a34e-fedf529692cb None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:05:53.446103 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45abe29f-3548-48b4-b95d-06241e9a70fa None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:05:53.481897 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e66f2f2-fce4-453b-b41e-ad465dadfbcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:05:53.482689 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e66f2f2-fce4-453b-b41e-ad465dadfbcd None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:53.482972 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e66f2f2-fce4-453b-b41e-ad465dadfbcd None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:53.483247 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e66f2f2-fce4-453b-b41e-ad465dadfbcd None admin] PATH_INFO: `/v3/domains/default` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:53.484437 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e66f2f2-fce4-453b-b41e-ad465dadfbcd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:05:53.485881 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e66f2f2-fce4-453b-b41e-ad465dadfbcd None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:05:53.487484 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 64/118] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jun 29 11:05:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:05:53.488266 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e66f2f2-fce4-453b-b41e-ad465dadfbcd None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:05:53.489793 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ff8fe1-45e6-4135-a34e-fedf529692cb None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:05:53.505748 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:05:53.523650 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96adf458-b81f-4ba7-9dec-b08625c58bcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:05:53.524539 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96adf458-b81f-4ba7-9dec-b08625c58bcb None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:53.524827 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96adf458-b81f-4ba7-9dec-b08625c58bcb None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:53.525106 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96adf458-b81f-4ba7-9dec-b08625c58bcb None admin] PATH_INFO: `/v3/projects/3b98d22914de4e3b8bca7fe0a0b51357` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:53.526296 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96adf458-b81f-4ba7-9dec-b08625c58bcb None admin] RBAC: Authorizing `identity:get_project(project_id=3b98d22914de4e3b8bca7fe0a0b51357)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:05:53.527730 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96adf458-b81f-4ba7-9dec-b08625c58bcb None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:05:53.529451 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 55/119] 10.222.0.44 () {56 vars in 1186 bytes} [Wed Jun 29 11:05:53 2022] GET /identity/v3/projects/3b98d22914de4e3b8bca7fe0a0b51357 => generated 298 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:05:53.533124 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb300747-4f10-45ee-a94c-e34438dc6915 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:53.533421 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb300747-4f10-45ee-a94c-e34438dc6915 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:53.533704 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb300747-4f10-45ee-a94c-e34438dc6915 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:53.539558 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 56/120] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:05:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:05:53.539558 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb300747-4f10-45ee-a94c-e34438dc6915 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:05:53.572847 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:05:53.588751 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e66f2f2-fce4-453b-b41e-ad465dadfbcd None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:05:53.621474 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069833da-52c1-4676-8148-a844b66985e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:05:53.622342 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069833da-52c1-4676-8148-a844b66985e2 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:53.622625 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069833da-52c1-4676-8148-a844b66985e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:53.622902 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069833da-52c1-4676-8148-a844b66985e2 None admin] PATH_INFO: `/v3/users/e2f3ce08c4a046fcae882fe888ca766d` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:53.624129 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069833da-52c1-4676-8148-a844b66985e2 None admin] RBAC: Authorizing `identity:get_user(user_id=e2f3ce08c4a046fcae882fe888ca766d)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:05:53.625585 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069833da-52c1-4676-8148-a844b66985e2 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:05:53.627199 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 65/121] 10.222.0.44 () {56 vars in 1177 bytes} [Wed Jun 29 11:05:53 2022] GET /identity/v3/users/e2f3ce08c4a046fcae882fe888ca766d => generated 275 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:05:53.627970 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069833da-52c1-4676-8148-a844b66985e2 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:05:53.640765 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb300747-4f10-45ee-a94c-e34438dc6915 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:05:53.673885 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5f8dea-6513-47e4-adbf-6df19fbcc37f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:05:53.674549 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5f8dea-6513-47e4-adbf-6df19fbcc37f None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:53.674836 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5f8dea-6513-47e4-adbf-6df19fbcc37f None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:53.675118 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5f8dea-6513-47e4-adbf-6df19fbcc37f None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:53.675881 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5f8dea-6513-47e4-adbf-6df19fbcc37f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:05:53.676193 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5f8dea-6513-47e4-adbf-6df19fbcc37f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:05:53.677564 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5f8dea-6513-47e4-adbf-6df19fbcc37f None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:05:53.686005 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 57/122] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Jun 29 11:05:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:05:53.686609 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5f8dea-6513-47e4-adbf-6df19fbcc37f None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:05:53.731433 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069833da-52c1-4676-8148-a844b66985e2 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:05:53.732351 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:05:53.764687 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c20364-ad76-4f7e-b3d9-fceb7858d730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:05:53.765539 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c20364-ad76-4f7e-b3d9-fceb7858d730 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:53.765819 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c20364-ad76-4f7e-b3d9-fceb7858d730 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:53.766091 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c20364-ad76-4f7e-b3d9-fceb7858d730 None admin] PATH_INFO: `/v3/projects/3b98d22914de4e3b8bca7fe0a0b51357` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:53.767267 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c20364-ad76-4f7e-b3d9-fceb7858d730 None admin] RBAC: Authorizing `identity:get_project(project_id=3b98d22914de4e3b8bca7fe0a0b51357)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:05:53.768849 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c20364-ad76-4f7e-b3d9-fceb7858d730 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:05:53.770579 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 66/123] 10.222.0.44 () {56 vars in 1186 bytes} [Wed Jun 29 11:05:53 2022] GET /identity/v3/projects/3b98d22914de4e3b8bca7fe0a0b51357 => generated 298 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:05:53.771309 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c20364-ad76-4f7e-b3d9-fceb7858d730 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:05:53.784949 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5f8dea-6513-47e4-adbf-6df19fbcc37f None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:05:53.793315 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:05:53.817764 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71da3d30-3f8b-4d88-a2f4-82bec9a36551 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:05:53.818419 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71da3d30-3f8b-4d88-a2f4-82bec9a36551 None admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:53.818702 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71da3d30-3f8b-4d88-a2f4-82bec9a36551 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:53.818976 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71da3d30-3f8b-4d88-a2f4-82bec9a36551 None admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:53.820233 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71da3d30-3f8b-4d88-a2f4-82bec9a36551 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:05:53.821711 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71da3d30-3f8b-4d88-a2f4-82bec9a36551 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:05:53.870806 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c20364-ad76-4f7e-b3d9-fceb7858d730 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:05:53.903538 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19b61d6-e7f9-411a-b64c-c253e0669306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:05:53.904440 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19b61d6-e7f9-411a-b64c-c253e0669306 None admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:53.904728 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19b61d6-e7f9-411a-b64c-c253e0669306 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:53.905013 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19b61d6-e7f9-411a-b64c-c253e0669306 None admin] PATH_INFO: `/v3/projects/3b98d22914de4e3b8bca7fe0a0b51357/users/e2f3ce08c4a046fcae882fe888ca766d/roles/e1c88eebe911432cab6a07fbeb84653b` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:53.907011 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19b61d6-e7f9-411a-b64c-c253e0669306 None admin] RBAC: Authorizing `identity:create_grant(project_id=3b98d22914de4e3b8bca7fe0a0b51357, user_id=e2f3ce08c4a046fcae882fe888ca766d, role_id=e1c88eebe911432cab6a07fbeb84653b)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:05:53.908462 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19b61d6-e7f9-411a-b64c-c253e0669306 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:05:53.908890 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 58/124] 10.222.0.44 () {60 vars in 1141 bytes} [Wed Jun 29 11:05:53 2022] POST /identity/v3/projects => generated 302 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:05:53.910147 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162aa889-809b-4696-982c-0cf8a42524f1 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:53.911312 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162aa889-809b-4696-982c-0cf8a42524f1 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:53.911594 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162aa889-809b-4696-982c-0cf8a42524f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:53.934063 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-162aa889-809b-4696-982c-0cf8a42524f1 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:05:53.935401 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-162aa889-809b-4696-982c-0cf8a42524f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:05:53.978256 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 67/125] 10.222.0.44 () {58 vars in 1439 bytes} [Wed Jun 29 11:05:53 2022] PUT /identity/v3/projects/3b98d22914de4e3b8bca7fe0a0b51357/users/e2f3ce08c4a046fcae882fe888ca766d/roles/e1c88eebe911432cab6a07fbeb84653b => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:05:53.979397 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19b61d6-e7f9-411a-b64c-c253e0669306 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:05:54.017477 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 59/126] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:05:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 11:05:54.018697 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162aa889-809b-4696-982c-0cf8a42524f1 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:05:54.052674 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:05:54.112414 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:05:54.138001 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162aa889-809b-4696-982c-0cf8a42524f1 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:05:54.140350 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19b61d6-e7f9-411a-b64c-c253e0669306 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:05:54.176465 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce05722a-9217-403a-b2cb-3cd196e1eded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:05:54.177344 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce05722a-9217-403a-b2cb-3cd196e1eded None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:54.179798 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b578a216-9519-4f44-83c4-e19591d918ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:05:54.179972 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce05722a-9217-403a-b2cb-3cd196e1eded None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:54.179972 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce05722a-9217-403a-b2cb-3cd196e1eded None admin] PATH_INFO: `/v3/projects/3b98d22914de4e3b8bca7fe0a0b51357` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:54.180115 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b578a216-9519-4f44-83c4-e19591d918ed None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:54.180115 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b578a216-9519-4f44-83c4-e19591d918ed None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:54.180115 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b578a216-9519-4f44-83c4-e19591d918ed None admin] PATH_INFO: `/v3/users/cdb4139035bd4d46a35dca750b630776` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:54.181189 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce05722a-9217-403a-b2cb-3cd196e1eded None admin] RBAC: Authorizing `identity:get_project(project_id=3b98d22914de4e3b8bca7fe0a0b51357)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:05:54.181291 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b578a216-9519-4f44-83c4-e19591d918ed None admin] RBAC: Authorizing `identity:get_user(user_id=cdb4139035bd4d46a35dca750b630776)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:05:54.182635 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b578a216-9519-4f44-83c4-e19591d918ed None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:05:54.182734 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce05722a-9217-403a-b2cb-3cd196e1eded None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:05:54.184318 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 60/127] 10.222.0.44 () {56 vars in 1177 bytes} [Wed Jun 29 11:05:54 2022] GET /identity/v3/users/cdb4139035bd4d46a35dca750b630776 => generated 247 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:05:54.184421 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 68/128] 10.222.0.44 () {56 vars in 1186 bytes} [Wed Jun 29 11:05:53 2022] GET /identity/v3/projects/3b98d22914de4e3b8bca7fe0a0b51357 => generated 298 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:05:54.185215 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce05722a-9217-403a-b2cb-3cd196e1eded None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:05:54.185326 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b578a216-9519-4f44-83c4-e19591d918ed None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:05:54.295399 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:05:54.301268 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b578a216-9519-4f44-83c4-e19591d918ed None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:05:54.323213 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce05722a-9217-403a-b2cb-3cd196e1eded None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:05:54.340902 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b02f52-01d1-4563-ad78-70d82b6c87dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:05:54.341570 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b02f52-01d1-4563-ad78-70d82b6c87dd None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:54.341861 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b02f52-01d1-4563-ad78-70d82b6c87dd None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:54.342137 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b02f52-01d1-4563-ad78-70d82b6c87dd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:54.342979 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b02f52-01d1-4563-ad78-70d82b6c87dd None admin] RBAC: Adding query filter params (role.id=7416219b213f48a4a93d964e67f990fe, scope.project.id=3b98d22914de4e3b8bca7fe0a0b51357, user.id=c444598ebd8845d2a9cdf41cefa70e01) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:05:54.343324 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b02f52-01d1-4563-ad78-70d82b6c87dd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:05:54.344804 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b02f52-01d1-4563-ad78-70d82b6c87dd None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:05:54.350983 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 61/129] 10.222.0.44 () {56 vars in 1374 bytes} [Wed Jun 29 11:05:54 2022] GET /identity/v3/role_assignments?user.id=c444598ebd8845d2a9cdf41cefa70e01&scope.project.id=3b98d22914de4e3b8bca7fe0a0b51357&role.id=7416219b213f48a4a93d964e67f990fe => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:05:54.351778 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b02f52-01d1-4563-ad78-70d82b6c87dd None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:05:54.374734 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f920d5-5249-4f73-9185-8b126f7f566a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:05:54.376505 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f920d5-5249-4f73-9185-8b126f7f566a None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:54.377015 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f920d5-5249-4f73-9185-8b126f7f566a None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:54.377810 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f920d5-5249-4f73-9185-8b126f7f566a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:54.396512 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f920d5-5249-4f73-9185-8b126f7f566a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 11:05:54.408288 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-a8f920d5-5249-4f73-9185-8b126f7f566a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 29 11:05:54.408288 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:05:54.408288 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:05:54.408288 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:05:54.408288 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:05:54.408288 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:05:54.408288 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:05:54.408288 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:05:54.408288 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:05:54.408288 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:05:54.408288 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:05:54.408288 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:05:54.408288 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 11:05:54.408288 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 11:05:54.408288 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 11:05:54.408288 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 11:05:54.409243 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 11:05:54.409243 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 11:05:54.409243 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 11:05:54.409243 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 11:05:54.409243 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:05:54.409243 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:05:54.409243 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:05:54.409243 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:05:54.409243 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:05:54.409243 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:05:54.409243 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:05:54.409243 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jun 29 11:05:54.409243 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:05:54.409243 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:05:54.409243 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:05:54.409243 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:05:54.410006 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:05:54.410006 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:05:54.410006 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:05:54.410006 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:05:54.410006 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 11:05:54.410006 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 11:05:54.410006 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 11:05:54.410006 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 11:05:54.410006 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 11:05:54.410006 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 11:05:54.410006 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 29 11:05:54.410006 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jun 29 11:05:54.413434 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 69/130] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Jun 29 11:05:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:05:54.414464 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f920d5-5249-4f73-9185-8b126f7f566a None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:05:54.446898 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:05:54.484664 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b02f52-01d1-4563-ad78-70d82b6c87dd None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:05:54.523386 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65959f19-046a-4515-bb89-486d0c381254 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:05:54.524338 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65959f19-046a-4515-bb89-486d0c381254 None admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:54.524628 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65959f19-046a-4515-bb89-486d0c381254 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:54.524905 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65959f19-046a-4515-bb89-486d0c381254 None admin] PATH_INFO: `/v3/projects/3b98d22914de4e3b8bca7fe0a0b51357/users/e2f3ce08c4a046fcae882fe888ca766d/roles/fc91ce4ad4a44cc3b05d0936754e87ec` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:54.527261 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65959f19-046a-4515-bb89-486d0c381254 None admin] RBAC: Authorizing `identity:create_grant(project_id=3b98d22914de4e3b8bca7fe0a0b51357, user_id=e2f3ce08c4a046fcae882fe888ca766d, role_id=fc91ce4ad4a44cc3b05d0936754e87ec)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:05:54.528796 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65959f19-046a-4515-bb89-486d0c381254 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:05:54.607738 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 62/131] 10.222.0.44 () {58 vars in 1439 bytes} [Wed Jun 29 11:05:54 2022] PUT /identity/v3/projects/3b98d22914de4e3b8bca7fe0a0b51357/users/e2f3ce08c4a046fcae882fe888ca766d/roles/fc91ce4ad4a44cc3b05d0936754e87ec => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:05:54.608614 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65959f19-046a-4515-bb89-486d0c381254 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:05:54.628194 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f920d5-5249-4f73-9185-8b126f7f566a None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:05:54.633354 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:05:54.680460 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ad6470-3fd8-45ab-a22a-c2606452b3df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:05:54.681402 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ad6470-3fd8-45ab-a22a-c2606452b3df None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:54.681690 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ad6470-3fd8-45ab-a22a-c2606452b3df None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:54.682036 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ad6470-3fd8-45ab-a22a-c2606452b3df None admin] PATH_INFO: `/v3/projects/3b98d22914de4e3b8bca7fe0a0b51357` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:54.684596 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ad6470-3fd8-45ab-a22a-c2606452b3df None admin] RBAC: Authorizing `identity:get_project(project_id=3b98d22914de4e3b8bca7fe0a0b51357)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:05:54.686103 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ad6470-3fd8-45ab-a22a-c2606452b3df None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:05:54.692409 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 70/132] 10.222.0.44 () {56 vars in 1186 bytes} [Wed Jun 29 11:05:54 2022] GET /identity/v3/projects/3b98d22914de4e3b8bca7fe0a0b51357 => generated 298 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:05:54.698607 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ad6470-3fd8-45ab-a22a-c2606452b3df None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:05:54.840200 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:05:54.849615 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65959f19-046a-4515-bb89-486d0c381254 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:05:54.872845 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ad6470-3fd8-45ab-a22a-c2606452b3df None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:05:54.895713 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a2f122-9eaa-4407-be7e-c1bf566432d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:05:54.896587 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a2f122-9eaa-4407-be7e-c1bf566432d1 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:54.896877 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a2f122-9eaa-4407-be7e-c1bf566432d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:54.897534 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a2f122-9eaa-4407-be7e-c1bf566432d1 None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:54.898357 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a2f122-9eaa-4407-be7e-c1bf566432d1 None admin] RBAC: Adding query filter params (name=member) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:05:54.898667 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a2f122-9eaa-4407-be7e-c1bf566432d1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:05:54.900091 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a2f122-9eaa-4407-be7e-c1bf566432d1 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:05:54.910880 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 63/133] 10.222.0.44 () {56 vars in 1101 bytes} [Wed Jun 29 11:05:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:05:54.919798 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a2f122-9eaa-4407-be7e-c1bf566432d1 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:05:54.942482 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c1eb95-1e90-4886-af89-773cc336f905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:05:54.943494 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c1eb95-1e90-4886-af89-773cc336f905 None admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:54.943781 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c1eb95-1e90-4886-af89-773cc336f905 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:54.944090 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c1eb95-1e90-4886-af89-773cc336f905 None admin] PATH_INFO: `/v3/projects/3b98d22914de4e3b8bca7fe0a0b51357/users/cdb4139035bd4d46a35dca750b630776/roles/e95c2ce63e114cc19a9dda6ad21e794f` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:54.946251 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c1eb95-1e90-4886-af89-773cc336f905 None admin] RBAC: Authorizing `identity:create_grant(project_id=3b98d22914de4e3b8bca7fe0a0b51357, user_id=cdb4139035bd4d46a35dca750b630776, role_id=e95c2ce63e114cc19a9dda6ad21e794f)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:05:54.951553 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c1eb95-1e90-4886-af89-773cc336f905 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:05:55.016943 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 71/134] 10.222.0.44 () {58 vars in 1439 bytes} [Wed Jun 29 11:05:54 2022] PUT /identity/v3/projects/3b98d22914de4e3b8bca7fe0a0b51357/users/cdb4139035bd4d46a35dca750b630776/roles/e95c2ce63e114cc19a9dda6ad21e794f => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 11:05:55.033893 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:05:55.055414 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a2f122-9eaa-4407-be7e-c1bf566432d1 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:05:55.093314 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9751cc87-77c3-4a55-b0e2-c943ec99e235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:05:55.094663 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9751cc87-77c3-4a55-b0e2-c943ec99e235 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:55.094951 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9751cc87-77c3-4a55-b0e2-c943ec99e235 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:55.095230 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9751cc87-77c3-4a55-b0e2-c943ec99e235 None admin] PATH_INFO: `/v3/users/e2f3ce08c4a046fcae882fe888ca766d` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:55.096530 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9751cc87-77c3-4a55-b0e2-c943ec99e235 None admin] RBAC: Authorizing `identity:get_user(user_id=e2f3ce08c4a046fcae882fe888ca766d)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:05:55.098029 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9751cc87-77c3-4a55-b0e2-c943ec99e235 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:05:55.099732 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 64/135] 10.222.0.44 () {56 vars in 1177 bytes} [Wed Jun 29 11:05:54 2022] GET /identity/v3/users/e2f3ce08c4a046fcae882fe888ca766d => generated 275 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:05:55.106011 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c1eb95-1e90-4886-af89-773cc336f905 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:05:55.216005 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c1eb95-1e90-4886-af89-773cc336f905 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:05:55.251840 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69746672-fa3e-4947-a5dd-1c1b9594fe1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:05:55.252831 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69746672-fa3e-4947-a5dd-1c1b9594fe1c None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:55.253186 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69746672-fa3e-4947-a5dd-1c1b9594fe1c None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:55.253477 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69746672-fa3e-4947-a5dd-1c1b9594fe1c None admin] PATH_INFO: `/v3/projects/89118c554be440f7b5287f08494d31b6` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:55.254765 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69746672-fa3e-4947-a5dd-1c1b9594fe1c None admin] RBAC: Authorizing `identity:get_project(project_id=89118c554be440f7b5287f08494d31b6)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:05:55.259871 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69746672-fa3e-4947-a5dd-1c1b9594fe1c None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:05:55.259871 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 72/136] 10.222.0.44 () {56 vars in 1186 bytes} [Wed Jun 29 11:05:55 2022] GET /identity/v3/projects/89118c554be440f7b5287f08494d31b6 => generated 312 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:05:55.266182 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9751cc87-77c3-4a55-b0e2-c943ec99e235 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:05:55.380904 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9751cc87-77c3-4a55-b0e2-c943ec99e235 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:05:55.423377 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46418dca-befa-4b83-8142-f6a1d6fd1b66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:05:55.424572 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46418dca-befa-4b83-8142-f6a1d6fd1b66 None admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:55.425003 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46418dca-befa-4b83-8142-f6a1d6fd1b66 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:55.425425 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46418dca-befa-4b83-8142-f6a1d6fd1b66 None admin] PATH_INFO: `/v3/projects/89118c554be440f7b5287f08494d31b6/users/e2f3ce08c4a046fcae882fe888ca766d/roles/e1c88eebe911432cab6a07fbeb84653b` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:55.427621 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46418dca-befa-4b83-8142-f6a1d6fd1b66 None admin] RBAC: Authorizing `identity:create_grant(project_id=89118c554be440f7b5287f08494d31b6, user_id=e2f3ce08c4a046fcae882fe888ca766d, role_id=e1c88eebe911432cab6a07fbeb84653b)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:05:55.429304 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46418dca-befa-4b83-8142-f6a1d6fd1b66 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:05:55.509208 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 65/137] 10.222.0.44 () {58 vars in 1439 bytes} [Wed Jun 29 11:05:55 2022] PUT /identity/v3/projects/89118c554be440f7b5287f08494d31b6/users/e2f3ce08c4a046fcae882fe888ca766d/roles/e1c88eebe911432cab6a07fbeb84653b => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:05:56.508906 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6d8667-5235-4989-9dc8-625b9a0c42ed None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:56.509426 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6d8667-5235-4989-9dc8-625b9a0c42ed None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:56.509498 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6d8667-5235-4989-9dc8-625b9a0c42ed None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:56.510860 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 73/138] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:05:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:05:56.516189 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f1bd1b-f893-4430-999e-a6eb0b1ce666 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:56.516586 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f1bd1b-f893-4430-999e-a6eb0b1ce666 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:56.516766 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f1bd1b-f893-4430-999e-a6eb0b1ce666 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:56.540866 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-34f1bd1b-f893-4430-999e-a6eb0b1ce666 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:05:56.542103 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-34f1bd1b-f893-4430-999e-a6eb0b1ce666 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:05:56.545221 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:05:56.626155 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 66/139] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:05:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 11:05:56.675140 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6d8667-5235-4989-9dc8-625b9a0c42ed None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:05:56.679675 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:05:56.809632 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6d8667-5235-4989-9dc8-625b9a0c42ed None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:05:56.853453 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dafb679-4091-419e-acd2-f7883f89b267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:05:56.854255 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dafb679-4091-419e-acd2-f7883f89b267 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:56.854545 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dafb679-4091-419e-acd2-f7883f89b267 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:56.854835 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dafb679-4091-419e-acd2-f7883f89b267 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:56.856045 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dafb679-4091-419e-acd2-f7883f89b267 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:05:56.857495 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dafb679-4091-419e-acd2-f7883f89b267 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:05:56.859116 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 74/140] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jun 29 11:05:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:05:56.864553 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f1bd1b-f893-4430-999e-a6eb0b1ce666 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:05:56.967295 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f1bd1b-f893-4430-999e-a6eb0b1ce666 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:05:57.007904 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59a5bc6-6c16-41bc-9840-164ccfc3fb87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:05:57.008599 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59a5bc6-6c16-41bc-9840-164ccfc3fb87 None admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:57.008890 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59a5bc6-6c16-41bc-9840-164ccfc3fb87 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:57.009175 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59a5bc6-6c16-41bc-9840-164ccfc3fb87 None admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:57.009918 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59a5bc6-6c16-41bc-9840-164ccfc3fb87 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:05:57.011366 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59a5bc6-6c16-41bc-9840-164ccfc3fb87 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:05:57.030219 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:05:57.134249 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ccf8d8-5503-4236-9fdf-bb822b8b5d46 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:57.135747 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ccf8d8-5503-4236-9fdf-bb822b8b5d46 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:57.135747 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ccf8d8-5503-4236-9fdf-bb822b8b5d46 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:57.136935 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 75/141] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:05:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:05:57.141939 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caadf118-5173-4933-a3cd-05b297c637f8 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:57.142229 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caadf118-5173-4933-a3cd-05b297c637f8 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:57.142510 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caadf118-5173-4933-a3cd-05b297c637f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:57.166496 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 67/142] 10.222.0.44 () {60 vars in 1132 bytes} [Wed Jun 29 11:05:56 2022] POST /identity/v3/users => generated 283 bytes in 302 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:05:57.175781 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-caadf118-5173-4933-a3cd-05b297c637f8 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:05:57.177030 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-caadf118-5173-4933-a3cd-05b297c637f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:05:57.275268 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 76/143] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:05:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 29 11:05:57.311551 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59a5bc6-6c16-41bc-9840-164ccfc3fb87 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:05:57.330944 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4246021d-f246-4a94-80ab-a144acd1d7d5 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:57.331231 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4246021d-f246-4a94-80ab-a144acd1d7d5 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:57.331514 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4246021d-f246-4a94-80ab-a144acd1d7d5 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:57.332954 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 77/144] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:05:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:05:57.339128 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d55245d-2e8f-4659-a011-82bdad59b28b None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:57.339418 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d55245d-2e8f-4659-a011-82bdad59b28b None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:57.339703 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d55245d-2e8f-4659-a011-82bdad59b28b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:57.344927 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:05:57.357846 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-4d55245d-2e8f-4659-a011-82bdad59b28b None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:05:57.359317 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-4d55245d-2e8f-4659-a011-82bdad59b28b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:05:57.456116 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59a5bc6-6c16-41bc-9840-164ccfc3fb87 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:05:57.490814 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 78/145] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:05:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 29 11:05:57.497975 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0e1b46-4983-49ae-86ed-06ef6bf6e110 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:57.497975 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0e1b46-4983-49ae-86ed-06ef6bf6e110 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:57.497975 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0e1b46-4983-49ae-86ed-06ef6bf6e110 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:57.499347 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 79/146] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:05:57 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:05:57.504487 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8f0256-e6c8-4d6a-b472-417c7cca0550 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:57.504778 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8f0256-e6c8-4d6a-b472-417c7cca0550 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:57.505059 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8f0256-e6c8-4d6a-b472-417c7cca0550 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:57.513447 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c04780e-3b50-4cd8-99d9-115f3471dc9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:05:57.514719 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c04780e-3b50-4cd8-99d9-115f3471dc9b None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:57.517549 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c04780e-3b50-4cd8-99d9-115f3471dc9b None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:57.518489 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c04780e-3b50-4cd8-99d9-115f3471dc9b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:57.534582 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-8f8f0256-e6c8-4d6a-b472-417c7cca0550 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:05:57.535788 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-8f8f0256-e6c8-4d6a-b472-417c7cca0550 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:05:57.537315 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c04780e-3b50-4cd8-99d9-115f3471dc9b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 11:05:57.548141 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-5c04780e-3b50-4cd8-99d9-115f3471dc9b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 29 11:05:57.548141 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:05:57.548141 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:05:57.548141 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:05:57.548141 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:05:57.548141 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:05:57.548141 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:05:57.548141 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:05:57.548141 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:05:57.548141 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:05:57.548141 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:05:57.548141 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:05:57.548141 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 11:05:57.548141 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 11:05:57.548141 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 11:05:57.548141 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 11:05:57.549015 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 11:05:57.549015 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 11:05:57.549015 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 11:05:57.549015 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 11:05:57.549015 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:05:57.549015 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:05:57.549015 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:05:57.549015 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:05:57.549015 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:05:57.549015 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:05:57.549015 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:05:57.549015 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jun 29 11:05:57.549015 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:05:57.549015 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:05:57.549015 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:05:57.549015 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:05:57.549769 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:05:57.549769 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:05:57.549769 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:05:57.549769 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:05:57.549769 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 11:05:57.549769 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 11:05:57.549769 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 11:05:57.549769 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 11:05:57.549769 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 11:05:57.549769 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 11:05:57.549769 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 29 11:05:57.549769 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:05:57.554480 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 68/147] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Jun 29 11:05:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:05:57.555950 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c04780e-3b50-4cd8-99d9-115f3471dc9b None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:05:57.626968 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 80/148] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:05:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 29 11:05:57.628325 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8f0256-e6c8-4d6a-b472-417c7cca0550 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:05:57.638529 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:05:57.693088 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c04780e-3b50-4cd8-99d9-115f3471dc9b None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:05:57.703621 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:05:57.741590 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a97cf51-39f6-4ba9-be6d-84583d9923a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:05:57.743025 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a97cf51-39f6-4ba9-be6d-84583d9923a0 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:57.743562 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a97cf51-39f6-4ba9-be6d-84583d9923a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:57.744043 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a97cf51-39f6-4ba9-be6d-84583d9923a0 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:57.756018 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a97cf51-39f6-4ba9-be6d-84583d9923a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 11:05:57.761080 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8f0256-e6c8-4d6a-b472-417c7cca0550 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:05:57.765007 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-2a97cf51-39f6-4ba9-be6d-84583d9923a0 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 29 11:05:57.765007 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:05:57.765007 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:05:57.765007 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:05:57.765007 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:05:57.765007 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:05:57.765007 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:05:57.765007 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:05:57.765007 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:05:57.765007 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:05:57.765007 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:05:57.765007 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:05:57.765007 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 11:05:57.765007 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 11:05:57.765007 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 11:05:57.765007 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 11:05:57.766061 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 11:05:57.766061 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 11:05:57.766061 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 11:05:57.766061 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 11:05:57.766061 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:05:57.766061 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:05:57.766061 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:05:57.766061 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:05:57.766061 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:05:57.766061 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:05:57.766061 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:05:57.766061 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jun 29 11:05:57.766061 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:05:57.766061 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:05:57.766061 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:05:57.766061 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:05:57.767116 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:05:57.767116 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:05:57.767116 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:05:57.767116 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:05:57.767116 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 11:05:57.767116 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 11:05:57.767116 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 11:05:57.767116 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 11:05:57.767116 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 11:05:57.767116 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 11:05:57.767116 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 29 11:05:57.767116 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:05:57.769302 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 69/149] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Jun 29 11:05:57 2022] GET /identity/v3/roles/reader => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:05:57.775581 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a97cf51-39f6-4ba9-be6d-84583d9923a0 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:05:57.803260 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26cfde7-c88a-4605-a71a-be717535253a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:05:57.804101 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26cfde7-c88a-4605-a71a-be717535253a None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:57.804482 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26cfde7-c88a-4605-a71a-be717535253a None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:57.804858 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26cfde7-c88a-4605-a71a-be717535253a None admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:57.805717 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26cfde7-c88a-4605-a71a-be717535253a None admin] RBAC: Adding query filter params (name=member) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:05:57.808094 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26cfde7-c88a-4605-a71a-be717535253a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:05:57.808094 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26cfde7-c88a-4605-a71a-be717535253a None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:05:57.816626 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 81/150] 10.222.0.44 () {56 vars in 1101 bytes} [Wed Jun 29 11:05:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:05:57.817554 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26cfde7-c88a-4605-a71a-be717535253a None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:05:57.903793 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a97cf51-39f6-4ba9-be6d-84583d9923a0 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:05:57.922676 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:05:57.932416 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26cfde7-c88a-4605-a71a-be717535253a None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:05:57.943428 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0363729-7b74-4794-8b14-4c0745456aad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:05:57.944969 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0363729-7b74-4794-8b14-4c0745456aad None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:57.945320 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0363729-7b74-4794-8b14-4c0745456aad None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:57.945609 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0363729-7b74-4794-8b14-4c0745456aad None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:57.958889 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:05:57.960936 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0363729-7b74-4794-8b14-4c0745456aad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 11:05:57.970277 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf7c49e-736f-4e56-8a11-538b129517e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:05:57.970544 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-f0363729-7b74-4794-8b14-4c0745456aad None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 29 11:05:57.970544 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:05:57.970544 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:05:57.970544 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:05:57.970544 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:05:57.970544 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:05:57.970544 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:05:57.970544 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:05:57.970544 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:05:57.970544 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:05:57.970544 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:05:57.970544 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:05:57.970544 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 11:05:57.970544 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 11:05:57.970544 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 11:05:57.970544 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 11:05:57.971355 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 11:05:57.971355 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 11:05:57.971355 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 11:05:57.971355 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 11:05:57.971355 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:05:57.971355 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:05:57.971355 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:05:57.971355 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:05:57.971355 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:05:57.971355 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:05:57.971355 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:05:57.971355 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jun 29 11:05:57.971355 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:05:57.971355 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:05:57.971355 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:05:57.971355 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:05:57.972572 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:05:57.972572 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:05:57.972572 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:05:57.972572 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:05:57.972572 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 11:05:57.972572 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 11:05:57.972572 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 11:05:57.972572 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 11:05:57.972572 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 11:05:57.972572 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 11:05:57.972572 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 29 11:05:57.972572 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:05:57.973297 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf7c49e-736f-4e56-8a11-538b129517e0 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:57.973297 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf7c49e-736f-4e56-8a11-538b129517e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:57.973297 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf7c49e-736f-4e56-8a11-538b129517e0 None admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:57.973464 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 70/151] 10.222.0.44 () {56 vars in 1114 bytes} [Wed Jun 29 11:05:57 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:05:57.973588 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf7c49e-736f-4e56-8a11-538b129517e0 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:05:57.973588 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf7c49e-736f-4e56-8a11-538b129517e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:05:57.973716 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0363729-7b74-4794-8b14-4c0745456aad None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:05:57.974045 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf7c49e-736f-4e56-8a11-538b129517e0 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:05:57.988560 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 82/152] 10.222.0.44 () {56 vars in 1101 bytes} [Wed Jun 29 11:05:57 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:05:57.989516 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf7c49e-736f-4e56-8a11-538b129517e0 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:05:58.086263 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0363729-7b74-4794-8b14-4c0745456aad None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:05:58.105020 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf7c49e-736f-4e56-8a11-538b129517e0 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:05:58.122979 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a290e1-eac5-456c-9a9c-55dea032e366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:05:58.123982 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a290e1-eac5-456c-9a9c-55dea032e366 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:58.124274 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a290e1-eac5-456c-9a9c-55dea032e366 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:58.124553 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a290e1-eac5-456c-9a9c-55dea032e366 None admin] PATH_INFO: `/v3/users/e2f3ce08c4a046fcae882fe888ca766d` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:58.125798 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a290e1-eac5-456c-9a9c-55dea032e366 None admin] RBAC: Authorizing `identity:get_user(user_id=e2f3ce08c4a046fcae882fe888ca766d)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:05:58.127372 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a290e1-eac5-456c-9a9c-55dea032e366 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:05:58.129163 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 71/153] 10.222.0.44 () {56 vars in 1177 bytes} [Wed Jun 29 11:05:57 2022] GET /identity/v3/users/e2f3ce08c4a046fcae882fe888ca766d => generated 275 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:05:58.130051 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a290e1-eac5-456c-9a9c-55dea032e366 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:05:58.140589 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb120b78-8b38-42e8-b685-0d5c5b90a4f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:05:58.141312 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb120b78-8b38-42e8-b685-0d5c5b90a4f6 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:58.141632 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb120b78-8b38-42e8-b685-0d5c5b90a4f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:58.141909 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb120b78-8b38-42e8-b685-0d5c5b90a4f6 None admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:58.142667 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb120b78-8b38-42e8-b685-0d5c5b90a4f6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:05:58.142973 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb120b78-8b38-42e8-b685-0d5c5b90a4f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:05:58.144543 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb120b78-8b38-42e8-b685-0d5c5b90a4f6 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:05:58.154542 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 83/154] 10.222.0.44 () {56 vars in 1111 bytes} [Wed Jun 29 11:05:57 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:05:58.155502 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb120b78-8b38-42e8-b685-0d5c5b90a4f6 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:05:58.185467 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:05:58.214088 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:05:58.245707 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a290e1-eac5-456c-9a9c-55dea032e366 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:05:58.268860 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb120b78-8b38-42e8-b685-0d5c5b90a4f6 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:05:58.283745 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6110afc-9ea8-4d86-bc62-8e38cbd7f5b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:05:58.284972 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6110afc-9ea8-4d86-bc62-8e38cbd7f5b4 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:58.285401 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6110afc-9ea8-4d86-bc62-8e38cbd7f5b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:58.285820 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6110afc-9ea8-4d86-bc62-8e38cbd7f5b4 None admin] PATH_INFO: `/v3/users/c444598ebd8845d2a9cdf41cefa70e01` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:58.287301 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6110afc-9ea8-4d86-bc62-8e38cbd7f5b4 None admin] RBAC: Authorizing `identity:get_user(user_id=c444598ebd8845d2a9cdf41cefa70e01)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:05:58.289669 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6110afc-9ea8-4d86-bc62-8e38cbd7f5b4 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:05:58.291592 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 72/155] 10.222.0.44 () {56 vars in 1177 bytes} [Wed Jun 29 11:05:58 2022] GET /identity/v3/users/c444598ebd8845d2a9cdf41cefa70e01 => generated 289 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:05:58.292544 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6110afc-9ea8-4d86-bc62-8e38cbd7f5b4 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:05:58.305643 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dc1760-d884-4f85-bed0-79e1a497244c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:05:58.306576 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dc1760-d884-4f85-bed0-79e1a497244c None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:58.306751 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dc1760-d884-4f85-bed0-79e1a497244c None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:58.307032 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dc1760-d884-4f85-bed0-79e1a497244c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:58.383478 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33dc1760-d884-4f85-bed0-79e1a497244c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:05:58.383785 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33dc1760-d884-4f85-bed0-79e1a497244c None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:05:58.405995 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6110afc-9ea8-4d86-bc62-8e38cbd7f5b4 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:05:58.424231 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:05:58.434444 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:05:58.449232 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71814170-6e2d-4fe8-b3ce-a792b0ebdf5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:05:58.450343 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71814170-6e2d-4fe8-b3ce-a792b0ebdf5c None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:58.450805 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71814170-6e2d-4fe8-b3ce-a792b0ebdf5c None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:58.451228 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71814170-6e2d-4fe8-b3ce-a792b0ebdf5c None admin] PATH_INFO: `/v3/users/e2f3ce08c4a046fcae882fe888ca766d` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:58.452667 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71814170-6e2d-4fe8-b3ce-a792b0ebdf5c None admin] RBAC: Authorizing `identity:get_user(user_id=e2f3ce08c4a046fcae882fe888ca766d)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:05:58.454274 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71814170-6e2d-4fe8-b3ce-a792b0ebdf5c None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:05:58.456144 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 73/156] 10.222.0.44 () {56 vars in 1177 bytes} [Wed Jun 29 11:05:58 2022] GET /identity/v3/users/e2f3ce08c4a046fcae882fe888ca766d => generated 275 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:05:58.457067 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71814170-6e2d-4fe8-b3ce-a792b0ebdf5c None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:05:58.494572 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 84/157] 10.222.0.44 () {58 vars in 1282 bytes} [Wed Jun 29 11:05:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 29 11:05:58.495743 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dc1760-d884-4f85-bed0-79e1a497244c None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:05:58.572862 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71814170-6e2d-4fe8-b3ce-a792b0ebdf5c None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:05:58.609575 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dc1760-d884-4f85-bed0-79e1a497244c None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:05:58.612289 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb9ed1a-f369-4053-9e0b-eb08f86258cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:05:58.613408 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb9ed1a-f369-4053-9e0b-eb08f86258cc None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:58.613835 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb9ed1a-f369-4053-9e0b-eb08f86258cc None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:58.614281 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb9ed1a-f369-4053-9e0b-eb08f86258cc None admin] PATH_INFO: `/v3/projects/3b98d22914de4e3b8bca7fe0a0b51357` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:58.615670 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb9ed1a-f369-4053-9e0b-eb08f86258cc None admin] RBAC: Authorizing `identity:get_project(project_id=3b98d22914de4e3b8bca7fe0a0b51357)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:05:58.617358 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb9ed1a-f369-4053-9e0b-eb08f86258cc None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:05:58.619556 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 74/158] 10.222.0.44 () {56 vars in 1186 bytes} [Wed Jun 29 11:05:58 2022] GET /identity/v3/projects/3b98d22914de4e3b8bca7fe0a0b51357 => generated 298 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:05:58.620535 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb9ed1a-f369-4053-9e0b-eb08f86258cc None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:05:58.647444 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e34f68f-7945-4754-ac4a-f9e2777a3843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:05:58.648696 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e34f68f-7945-4754-ac4a-f9e2777a3843 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:58.648978 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e34f68f-7945-4754-ac4a-f9e2777a3843 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:58.649248 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e34f68f-7945-4754-ac4a-f9e2777a3843 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:58.668987 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:05:58.684196 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:05:58.725943 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e34f68f-7945-4754-ac4a-f9e2777a3843 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:05:58.727415 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e34f68f-7945-4754-ac4a-f9e2777a3843 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:05:58.733700 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb9ed1a-f369-4053-9e0b-eb08f86258cc None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:05:58.771114 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2291c485-1a4f-4e25-9abe-6cb9f3b27090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:05:58.772294 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2291c485-1a4f-4e25-9abe-6cb9f3b27090 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:58.772761 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2291c485-1a4f-4e25-9abe-6cb9f3b27090 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:58.773182 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2291c485-1a4f-4e25-9abe-6cb9f3b27090 None admin] PATH_INFO: `/v3/projects/3b98d22914de4e3b8bca7fe0a0b51357` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:58.774524 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2291c485-1a4f-4e25-9abe-6cb9f3b27090 None admin] RBAC: Authorizing `identity:get_project(project_id=3b98d22914de4e3b8bca7fe0a0b51357)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:05:58.776326 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2291c485-1a4f-4e25-9abe-6cb9f3b27090 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:05:58.778218 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 75/159] 10.222.0.44 () {56 vars in 1186 bytes} [Wed Jun 29 11:05:58 2022] GET /identity/v3/projects/3b98d22914de4e3b8bca7fe0a0b51357 => generated 298 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:05:58.779100 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2291c485-1a4f-4e25-9abe-6cb9f3b27090 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:05:58.834203 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 85/160] 10.222.0.44 () {58 vars in 1282 bytes} [Wed Jun 29 11:05:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 29 11:05:58.835571 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b2d20e-76c8-4f51-b1d7-fe287e71c1d8 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:58.838885 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b2d20e-76c8-4f51-b1d7-fe287e71c1d8 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:58.839224 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b2d20e-76c8-4f51-b1d7-fe287e71c1d8 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:58.840924 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 86/161] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:05:58 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:05:58.841702 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b2d20e-76c8-4f51-b1d7-fe287e71c1d8 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:05:58.890061 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2291c485-1a4f-4e25-9abe-6cb9f3b27090 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:05:58.898839 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:05:58.907345 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:05:58.927099 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326978ca-e168-43c4-b0ef-80d6d23d2925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:05:58.928241 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326978ca-e168-43c4-b0ef-80d6d23d2925 None admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:58.928671 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326978ca-e168-43c4-b0ef-80d6d23d2925 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:58.929665 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326978ca-e168-43c4-b0ef-80d6d23d2925 None admin] PATH_INFO: `/v3/projects/3b98d22914de4e3b8bca7fe0a0b51357/users/c444598ebd8845d2a9cdf41cefa70e01/roles/7416219b213f48a4a93d964e67f990fe` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:58.934085 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326978ca-e168-43c4-b0ef-80d6d23d2925 None admin] RBAC: Authorizing `identity:create_grant(project_id=3b98d22914de4e3b8bca7fe0a0b51357, user_id=c444598ebd8845d2a9cdf41cefa70e01, role_id=7416219b213f48a4a93d964e67f990fe)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:05:58.934085 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326978ca-e168-43c4-b0ef-80d6d23d2925 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:05:58.968596 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b2d20e-76c8-4f51-b1d7-fe287e71c1d8 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:05:59.006180 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f0ad91-e877-42b5-8575-f323bf08089d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:05:59.007561 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f0ad91-e877-42b5-8575-f323bf08089d None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:59.007879 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f0ad91-e877-42b5-8575-f323bf08089d None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:59.008157 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f0ad91-e877-42b5-8575-f323bf08089d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:59.009187 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f0ad91-e877-42b5-8575-f323bf08089d None admin] RBAC: Adding query filter params (role.id=e1c88eebe911432cab6a07fbeb84653b, scope.project.id=3b98d22914de4e3b8bca7fe0a0b51357, user.id=e2f3ce08c4a046fcae882fe888ca766d) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:05:59.009546 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f0ad91-e877-42b5-8575-f323bf08089d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:05:59.014331 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f0ad91-e877-42b5-8575-f323bf08089d None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:05:59.017995 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 76/163] 10.222.0.44 () {58 vars in 1439 bytes} [Wed Jun 29 11:05:58 2022] PUT /identity/v3/projects/3b98d22914de4e3b8bca7fe0a0b51357/users/c444598ebd8845d2a9cdf41cefa70e01/roles/7416219b213f48a4a93d964e67f990fe => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:05:59.018122 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 87/163] 10.222.0.44 () {56 vars in 1374 bytes} [Wed Jun 29 11:05:58 2022] GET /identity/v3/role_assignments?user.id=e2f3ce08c4a046fcae882fe888ca766d&scope.project.id=3b98d22914de4e3b8bca7fe0a0b51357&role.id=e1c88eebe911432cab6a07fbeb84653b => generated 608 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:05:59.019202 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326978ca-e168-43c4-b0ef-80d6d23d2925 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:05:59.019307 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976ac1cf-9b5b-4e24-8951-59535fd52712 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:59.019597 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976ac1cf-9b5b-4e24-8951-59535fd52712 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:59.019937 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976ac1cf-9b5b-4e24-8951-59535fd52712 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:59.039958 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-976ac1cf-9b5b-4e24-8951-59535fd52712 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:05:59.039958 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-976ac1cf-9b5b-4e24-8951-59535fd52712 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:05:59.120118 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 88/164] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:05:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 11:05:59.144666 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976ac1cf-9b5b-4e24-8951-59535fd52712 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:05:59.176342 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326978ca-e168-43c4-b0ef-80d6d23d2925 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:05:59.194048 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:05:59.245646 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807c256a-8e00-4111-b584-85a91adf8563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:05:59.246624 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807c256a-8e00-4111-b584-85a91adf8563 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:59.246913 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807c256a-8e00-4111-b584-85a91adf8563 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:59.247243 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807c256a-8e00-4111-b584-85a91adf8563 None admin] PATH_INFO: `/v3/projects/3b98d22914de4e3b8bca7fe0a0b51357` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:59.248992 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807c256a-8e00-4111-b584-85a91adf8563 None admin] RBAC: Authorizing `identity:get_project(project_id=3b98d22914de4e3b8bca7fe0a0b51357)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:05:59.250501 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807c256a-8e00-4111-b584-85a91adf8563 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:05:59.252378 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 77/165] 10.222.0.44 () {56 vars in 1186 bytes} [Wed Jun 29 11:05:59 2022] GET /identity/v3/projects/3b98d22914de4e3b8bca7fe0a0b51357 => generated 298 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:05:59.268622 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976ac1cf-9b5b-4e24-8951-59535fd52712 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:05:59.278613 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807c256a-8e00-4111-b584-85a91adf8563 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:05:59.312755 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3667417-9131-492d-82cd-f01a9cd74674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:05:59.314133 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3667417-9131-492d-82cd-f01a9cd74674 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:59.314621 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3667417-9131-492d-82cd-f01a9cd74674 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:59.315094 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3667417-9131-492d-82cd-f01a9cd74674 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:59.334784 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3667417-9131-492d-82cd-f01a9cd74674 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 11:05:59.345439 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-e3667417-9131-492d-82cd-f01a9cd74674 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 11:05:59.345439 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:05:59.345439 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:05:59.345439 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:05:59.345439 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:05:59.345439 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:05:59.345439 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:05:59.345439 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:05:59.345439 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:05:59.345439 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:05:59.345439 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:05:59.345439 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:05:59.345439 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 11:05:59.345439 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 11:05:59.345439 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 11:05:59.345439 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 11:05:59.346456 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 11:05:59.346456 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 11:05:59.346456 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 11:05:59.346456 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 11:05:59.346456 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:05:59.346456 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:05:59.346456 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:05:59.346456 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:05:59.346456 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:05:59.346456 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:05:59.346456 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:05:59.346456 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jun 29 11:05:59.346456 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:05:59.346456 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:05:59.346456 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:05:59.346456 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:05:59.352987 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:05:59.352987 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:05:59.352987 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:05:59.352987 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:05:59.352987 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 11:05:59.352987 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 11:05:59.352987 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 11:05:59.352987 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 11:05:59.352987 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 11:05:59.352987 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 11:05:59.352987 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 11:05:59.352987 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jun 29 11:05:59.355407 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 89/166] 10.222.0.44 () {56 vars in 1096 bytes} [Wed Jun 29 11:05:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 11:05:59.357036 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefe515b-9a1f-49f9-90d3-de2d989cfb62 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:59.357588 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefe515b-9a1f-49f9-90d3-de2d989cfb62 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:59.358067 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefe515b-9a1f-49f9-90d3-de2d989cfb62 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:59.359961 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 90/167] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:05:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:05:59.361654 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefe515b-9a1f-49f9-90d3-de2d989cfb62 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:05:59.401825 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:05:59.509112 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807c256a-8e00-4111-b584-85a91adf8563 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:05:59.551563 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefe515b-9a1f-49f9-90d3-de2d989cfb62 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:05:59.553694 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:05:59.586653 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ad7215-bc7c-4ced-93b3-26a7baf13699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:05:59.587400 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ad7215-bc7c-4ced-93b3-26a7baf13699 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:59.587929 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ad7215-bc7c-4ced-93b3-26a7baf13699 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:59.587929 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ad7215-bc7c-4ced-93b3-26a7baf13699 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:59.588896 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ad7215-bc7c-4ced-93b3-26a7baf13699 None admin] RBAC: Adding query filter params (role.id=fc91ce4ad4a44cc3b05d0936754e87ec, scope.project.id=3b98d22914de4e3b8bca7fe0a0b51357, user.id=e2f3ce08c4a046fcae882fe888ca766d) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:05:59.589222 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ad7215-bc7c-4ced-93b3-26a7baf13699 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:05:59.590753 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ad7215-bc7c-4ced-93b3-26a7baf13699 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:05:59.605261 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 78/168] 10.222.0.44 () {56 vars in 1374 bytes} [Wed Jun 29 11:05:59 2022] GET /identity/v3/role_assignments?user.id=e2f3ce08c4a046fcae882fe888ca766d&scope.project.id=3b98d22914de4e3b8bca7fe0a0b51357&role.id=fc91ce4ad4a44cc3b05d0936754e87ec => generated 608 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 11:05:59.606755 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9a2217-eddd-4586-b068-71bc923f7a9f None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:59.607054 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9a2217-eddd-4586-b068-71bc923f7a9f None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:59.607337 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9a2217-eddd-4586-b068-71bc923f7a9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:59.624709 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711c7a01-4359-41e1-8afc-c009d9d89093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:05:59.625517 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711c7a01-4359-41e1-8afc-c009d9d89093 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:59.625805 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711c7a01-4359-41e1-8afc-c009d9d89093 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:59.626083 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711c7a01-4359-41e1-8afc-c009d9d89093 None admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:59.626857 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711c7a01-4359-41e1-8afc-c009d9d89093 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:05:59.628950 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711c7a01-4359-41e1-8afc-c009d9d89093 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:05:59.630383 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711c7a01-4359-41e1-8afc-c009d9d89093 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:05:59.651819 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-5c9a2217-eddd-4586-b068-71bc923f7a9f None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:05:59.653441 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-5c9a2217-eddd-4586-b068-71bc923f7a9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:05:59.658203 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 91/169] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Jun 29 11:05:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:05:59.664996 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711c7a01-4359-41e1-8afc-c009d9d89093 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:05:59.854086 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711c7a01-4359-41e1-8afc-c009d9d89093 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:05:59.861573 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 79/170] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:05:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 11:05:59.905318 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9a2217-eddd-4586-b068-71bc923f7a9f None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:05:59.915597 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd19e511-cbbb-45fb-b275-1eaf4707ff5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:05:59.916699 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd19e511-cbbb-45fb-b275-1eaf4707ff5c None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:05:59.916699 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd19e511-cbbb-45fb-b275-1eaf4707ff5c None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:05:59.916875 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd19e511-cbbb-45fb-b275-1eaf4707ff5c None admin] PATH_INFO: `/v3/users/cdb4139035bd4d46a35dca750b630776` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:05:59.920606 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd19e511-cbbb-45fb-b275-1eaf4707ff5c None admin] RBAC: Authorizing `identity:get_user(user_id=cdb4139035bd4d46a35dca750b630776)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:05:59.928444 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd19e511-cbbb-45fb-b275-1eaf4707ff5c None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:05:59.928444 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 92/171] 10.222.0.44 () {56 vars in 1177 bytes} [Wed Jun 29 11:05:59 2022] GET /identity/v3/users/cdb4139035bd4d46a35dca750b630776 => generated 247 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 11:05:59.936899 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd19e511-cbbb-45fb-b275-1eaf4707ff5c None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:00.008467 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:00.017159 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:00.131107 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9a2217-eddd-4586-b068-71bc923f7a9f None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:00.151230 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd19e511-cbbb-45fb-b275-1eaf4707ff5c None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:00.193361 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74c560f-6f69-4d1a-962f-b323a137e97a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:00.194169 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74c560f-6f69-4d1a-962f-b323a137e97a None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:00.195001 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74c560f-6f69-4d1a-962f-b323a137e97a None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:00.195273 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74c560f-6f69-4d1a-962f-b323a137e97a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:00.218095 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f0109a-c787-4032-ae73-c0aba3cd6b7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:00.219092 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f0109a-c787-4032-ae73-c0aba3cd6b7b None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:00.219566 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f0109a-c787-4032-ae73-c0aba3cd6b7b None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:00.220161 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f0109a-c787-4032-ae73-c0aba3cd6b7b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:00.236518 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74c560f-6f69-4d1a-962f-b323a137e97a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 11:06:00.245593 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-d74c560f-6f69-4d1a-962f-b323a137e97a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 29 11:06:00.245593 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:06:00.245593 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:06:00.245593 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:06:00.245593 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:06:00.245593 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:06:00.245593 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:06:00.245593 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:06:00.245593 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:06:00.245593 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:06:00.245593 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:06:00.245593 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:06:00.245593 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 11:06:00.245593 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 11:06:00.245593 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 11:06:00.245593 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 11:06:00.246496 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 11:06:00.246496 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 11:06:00.246496 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 11:06:00.246496 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 11:06:00.246496 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:06:00.246496 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:06:00.246496 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:06:00.246496 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:06:00.246496 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:06:00.246496 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:06:00.246496 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:06:00.246496 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jun 29 11:06:00.246496 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:06:00.246496 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:06:00.246496 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:06:00.246496 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:06:00.247347 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:06:00.247347 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:06:00.247347 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:06:00.247347 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:06:00.247347 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 11:06:00.247347 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 11:06:00.247347 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 11:06:00.247347 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 11:06:00.247347 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 11:06:00.247347 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 11:06:00.247347 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 29 11:06:00.247347 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:06:00.247347 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 80/172] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Jun 29 11:05:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 342 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 11:06:00.257716 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74c560f-6f69-4d1a-962f-b323a137e97a None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:00.301879 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f0109a-c787-4032-ae73-c0aba3cd6b7b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:00.304271 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f0109a-c787-4032-ae73-c0aba3cd6b7b None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:00.340386 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:00.425698 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74c560f-6f69-4d1a-962f-b323a137e97a None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:00.428295 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:00.453141 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 93/173] 10.222.0.44 () {58 vars in 1282 bytes} [Wed Jun 29 11:05:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 517 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 29 11:06:00.458793 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f0109a-c787-4032-ae73-c0aba3cd6b7b None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:00.475385 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d34671-9b08-4ee0-841f-72bf60d21eb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:00.476154 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d34671-9b08-4ee0-841f-72bf60d21eb3 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:00.476448 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d34671-9b08-4ee0-841f-72bf60d21eb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:00.476732 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d34671-9b08-4ee0-841f-72bf60d21eb3 None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:00.477509 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d34671-9b08-4ee0-841f-72bf60d21eb3 None admin] RBAC: Adding query filter params (name=member) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:06:00.477824 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d34671-9b08-4ee0-841f-72bf60d21eb3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:00.479257 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d34671-9b08-4ee0-841f-72bf60d21eb3 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:00.502030 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 81/174] 10.222.0.44 () {56 vars in 1101 bytes} [Wed Jun 29 11:06:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:00.508616 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d34671-9b08-4ee0-841f-72bf60d21eb3 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:00.701591 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f0109a-c787-4032-ae73-c0aba3cd6b7b None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:00.739474 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d34671-9b08-4ee0-841f-72bf60d21eb3 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:00.785126 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:00.800503 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee781c91-f9b4-4ce4-bf1c-2fd05649dc83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:00.801382 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee781c91-f9b4-4ce4-bf1c-2fd05649dc83 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:00.801676 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee781c91-f9b4-4ce4-bf1c-2fd05649dc83 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:00.801960 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee781c91-f9b4-4ce4-bf1c-2fd05649dc83 None admin] PATH_INFO: `/v3/projects/3b98d22914de4e3b8bca7fe0a0b51357` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:00.803219 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee781c91-f9b4-4ce4-bf1c-2fd05649dc83 None admin] RBAC: Authorizing `identity:get_project(project_id=3b98d22914de4e3b8bca7fe0a0b51357)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:00.804730 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee781c91-f9b4-4ce4-bf1c-2fd05649dc83 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:00.806443 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 94/175] 10.222.0.44 () {56 vars in 1186 bytes} [Wed Jun 29 11:06:00 2022] GET /identity/v3/projects/3b98d22914de4e3b8bca7fe0a0b51357 => generated 298 bytes in 348 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 11:06:00.808585 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18360642-ba7e-4841-aa3d-92d11f2f1d44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:00.809484 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18360642-ba7e-4841-aa3d-92d11f2f1d44 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:00.809769 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18360642-ba7e-4841-aa3d-92d11f2f1d44 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:00.810042 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18360642-ba7e-4841-aa3d-92d11f2f1d44 None admin] PATH_INFO: `/v3/users/e2f3ce08c4a046fcae882fe888ca766d` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:00.811273 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18360642-ba7e-4841-aa3d-92d11f2f1d44 None admin] RBAC: Authorizing `identity:get_user(user_id=e2f3ce08c4a046fcae882fe888ca766d)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:00.812715 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18360642-ba7e-4841-aa3d-92d11f2f1d44 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:00.820214 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 82/176] 10.222.0.44 () {56 vars in 1177 bytes} [Wed Jun 29 11:06:00 2022] GET /identity/v3/users/e2f3ce08c4a046fcae882fe888ca766d => generated 275 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:00.820424 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee781c91-f9b4-4ce4-bf1c-2fd05649dc83 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:00.825455 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18360642-ba7e-4841-aa3d-92d11f2f1d44 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:00.884860 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:01.026962 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee781c91-f9b4-4ce4-bf1c-2fd05649dc83 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:01.089556 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f149ac01-ba9c-4c76-97f0-4d33307bb126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:01.090235 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f149ac01-ba9c-4c76-97f0-4d33307bb126 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:01.090523 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f149ac01-ba9c-4c76-97f0-4d33307bb126 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:01.090805 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f149ac01-ba9c-4c76-97f0-4d33307bb126 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:01.091653 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f149ac01-ba9c-4c76-97f0-4d33307bb126 None admin] RBAC: Adding query filter params (role.id=e95c2ce63e114cc19a9dda6ad21e794f, scope.project.id=3b98d22914de4e3b8bca7fe0a0b51357, user.id=cdb4139035bd4d46a35dca750b630776) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:06:01.091996 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f149ac01-ba9c-4c76-97f0-4d33307bb126 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:01.093455 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f149ac01-ba9c-4c76-97f0-4d33307bb126 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:01.099795 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 95/177] 10.222.0.44 () {56 vars in 1374 bytes} [Wed Jun 29 11:06:00 2022] GET /identity/v3/role_assignments?user.id=cdb4139035bd4d46a35dca750b630776&scope.project.id=3b98d22914de4e3b8bca7fe0a0b51357&role.id=e95c2ce63e114cc19a9dda6ad21e794f => generated 608 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 11:06:01.106117 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18360642-ba7e-4841-aa3d-92d11f2f1d44 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:01.175948 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26797c8b-a729-405f-a3aa-e304d9716c66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:01.176617 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26797c8b-a729-405f-a3aa-e304d9716c66 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:01.176902 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26797c8b-a729-405f-a3aa-e304d9716c66 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:01.177183 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26797c8b-a729-405f-a3aa-e304d9716c66 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:01.270187 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26797c8b-a729-405f-a3aa-e304d9716c66 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:01.271678 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26797c8b-a729-405f-a3aa-e304d9716c66 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:01.312662 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:01.402000 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 83/178] 10.222.0.44 () {58 vars in 1282 bytes} [Wed Jun 29 11:06:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 577 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 29 11:06:01.407812 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f149ac01-ba9c-4c76-97f0-4d33307bb126 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:01.446180 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db35c265-78a2-4c47-a5ed-0c5cfcb6d0dd None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:01.446802 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db35c265-78a2-4c47-a5ed-0c5cfcb6d0dd None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:01.450841 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db35c265-78a2-4c47-a5ed-0c5cfcb6d0dd None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:01.453317 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 84/179] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:06:01 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 29 11:06:01.461308 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0791c52-5e35-410b-bfa9-f8f1f2650b15 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:01.461868 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0791c52-5e35-410b-bfa9-f8f1f2650b15 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:01.462355 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0791c52-5e35-410b-bfa9-f8f1f2650b15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:01.491055 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-e0791c52-5e35-410b-bfa9-f8f1f2650b15 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:06:01.494265 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-e0791c52-5e35-410b-bfa9-f8f1f2650b15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:06:01.523729 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:01.569598 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f149ac01-ba9c-4c76-97f0-4d33307bb126 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:01.609729 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09efcec8-6ea4-4128-b7bd-e3b13f69aa66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:01.611081 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09efcec8-6ea4-4128-b7bd-e3b13f69aa66 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:01.612431 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09efcec8-6ea4-4128-b7bd-e3b13f69aa66 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:01.613555 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09efcec8-6ea4-4128-b7bd-e3b13f69aa66 None admin] PATH_INFO: `/v3/projects/89118c554be440f7b5287f08494d31b6` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:01.616882 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09efcec8-6ea4-4128-b7bd-e3b13f69aa66 None admin] RBAC: Authorizing `identity:get_project(project_id=89118c554be440f7b5287f08494d31b6)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:01.618750 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09efcec8-6ea4-4128-b7bd-e3b13f69aa66 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:01.621442 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 96/180] 10.222.0.44 () {56 vars in 1186 bytes} [Wed Jun 29 11:06:01 2022] GET /identity/v3/projects/89118c554be440f7b5287f08494d31b6 => generated 312 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:01.628879 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09efcec8-6ea4-4128-b7bd-e3b13f69aa66 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:01.634064 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 85/181] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:06:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 11:06:01.658899 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0791c52-5e35-410b-bfa9-f8f1f2650b15 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:01.722456 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:01.747923 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09efcec8-6ea4-4128-b7bd-e3b13f69aa66 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:01.768099 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0791c52-5e35-410b-bfa9-f8f1f2650b15 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:01.783010 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:01.792744 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb64ce6-5745-4dd0-aa37-d767d018520c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:01.796522 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb64ce6-5745-4dd0-aa37-d767d018520c None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:01.796522 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb64ce6-5745-4dd0-aa37-d767d018520c None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:01.796522 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb64ce6-5745-4dd0-aa37-d767d018520c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:01.796522 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb64ce6-5745-4dd0-aa37-d767d018520c None admin] RBAC: Adding query filter params (role.id=e1c88eebe911432cab6a07fbeb84653b, scope.project.id=89118c554be440f7b5287f08494d31b6, user.id=e2f3ce08c4a046fcae882fe888ca766d) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:06:01.796522 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb64ce6-5745-4dd0-aa37-d767d018520c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:01.797799 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb64ce6-5745-4dd0-aa37-d767d018520c None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:01.805540 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 97/182] 10.222.0.44 () {56 vars in 1374 bytes} [Wed Jun 29 11:06:01 2022] GET /identity/v3/role_assignments?user.id=e2f3ce08c4a046fcae882fe888ca766d&scope.project.id=89118c554be440f7b5287f08494d31b6&role.id=e1c88eebe911432cab6a07fbeb84653b => generated 608 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 11:06:01.807064 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b09cff0-33b1-455a-ad62-498fd3c59a3e None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:01.807545 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b09cff0-33b1-455a-ad62-498fd3c59a3e None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:01.808092 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b09cff0-33b1-455a-ad62-498fd3c59a3e None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:01.809967 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 98/183] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:06:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:06:01.813477 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7720b0bb-0453-4ef7-af03-7807093bf10b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:01.813477 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7720b0bb-0453-4ef7-af03-7807093bf10b None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:01.813477 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7720b0bb-0453-4ef7-af03-7807093bf10b None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:01.813477 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7720b0bb-0453-4ef7-af03-7807093bf10b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:01.816585 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64413488-985e-4225-b918-baefd408969b None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:01.817145 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64413488-985e-4225-b918-baefd408969b None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:01.817616 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64413488-985e-4225-b918-baefd408969b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:01.841103 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-64413488-985e-4225-b918-baefd408969b None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:06:01.842103 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7720b0bb-0453-4ef7-af03-7807093bf10b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 11:06:01.842238 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-64413488-985e-4225-b918-baefd408969b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:06:01.851003 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-7720b0bb-0453-4ef7-af03-7807093bf10b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 11:06:01.851003 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:06:01.851003 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:06:01.851003 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:06:01.851003 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:06:01.851003 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:06:01.851003 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:06:01.851003 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:06:01.851003 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:06:01.851003 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:06:01.851003 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:06:01.851003 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:06:01.851003 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 11:06:01.851003 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 11:06:01.851003 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 11:06:01.851003 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 11:06:01.852011 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 11:06:01.852011 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 11:06:01.852011 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 11:06:01.852011 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 11:06:01.852011 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:06:01.852011 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:06:01.852011 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:06:01.852011 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:06:01.852011 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:06:01.852011 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:06:01.852011 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:06:01.852011 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jun 29 11:06:01.852011 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:06:01.852011 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:06:01.852011 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:06:01.852011 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:06:01.852805 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:06:01.852805 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:06:01.852805 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:06:01.852805 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:06:01.852805 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 11:06:01.852805 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 11:06:01.852805 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 11:06:01.852805 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 11:06:01.852805 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 11:06:01.852805 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 11:06:01.852805 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 11:06:01.852805 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:06:01.855527 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 86/184] 10.222.0.44 () {56 vars in 1096 bytes} [Wed Jun 29 11:06:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 11:06:01.857616 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fc200f-61e0-428c-bfc0-bd46a93ac228 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:01.858167 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fc200f-61e0-428c-bfc0-bd46a93ac228 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:01.858695 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fc200f-61e0-428c-bfc0-bd46a93ac228 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:01.860357 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 87/185] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:06:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:06:01.874824 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fc200f-61e0-428c-bfc0-bd46a93ac228 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:01.932813 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 99/186] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:06:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 11:06:01.940204 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317b315a-826b-4b17-a71e-1c2f3b483cb4 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:01.940204 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317b315a-826b-4b17-a71e-1c2f3b483cb4 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:01.940204 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317b315a-826b-4b17-a71e-1c2f3b483cb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:01.964863 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-317b315a-826b-4b17-a71e-1c2f3b483cb4 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:06:01.966142 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-317b315a-826b-4b17-a71e-1c2f3b483cb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:06:02.010717 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fc200f-61e0-428c-bfc0-bd46a93ac228 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:02.033782 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:02.042081 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 100/187] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:06:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 11:06:02.042871 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317b315a-826b-4b17-a71e-1c2f3b483cb4 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:02.051269 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59f687e-4d99-4698-b28d-408673e1729f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:02.052303 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59f687e-4d99-4698-b28d-408673e1729f None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:02.052812 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59f687e-4d99-4698-b28d-408673e1729f None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:02.053299 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59f687e-4d99-4698-b28d-408673e1729f None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:02.054786 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59f687e-4d99-4698-b28d-408673e1729f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:06:02.055316 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59f687e-4d99-4698-b28d-408673e1729f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:02.057018 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59f687e-4d99-4698-b28d-408673e1729f None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:02.072191 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 88/188] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Jun 29 11:06:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:02.074540 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59f687e-4d99-4698-b28d-408673e1729f None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:02.106707 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:02.155505 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317b315a-826b-4b17-a71e-1c2f3b483cb4 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:02.181204 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59f687e-4d99-4698-b28d-408673e1729f None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:02.191134 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f253eb3-904f-4c4d-92fb-c8944936b1ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:02.191973 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f253eb3-904f-4c4d-92fb-c8944936b1ac None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:02.192256 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f253eb3-904f-4c4d-92fb-c8944936b1ac None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:02.192534 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f253eb3-904f-4c4d-92fb-c8944936b1ac None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:02.203203 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f253eb3-904f-4c4d-92fb-c8944936b1ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 11:06:02.211999 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-1f253eb3-904f-4c4d-92fb-c8944936b1ac None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 29 11:06:02.211999 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:06:02.211999 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:06:02.211999 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:06:02.211999 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:06:02.211999 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:06:02.211999 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:06:02.211999 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:06:02.211999 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:06:02.211999 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:06:02.211999 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:06:02.211999 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:06:02.211999 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 11:06:02.211999 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 11:06:02.211999 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 11:06:02.211999 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 11:06:02.213077 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 11:06:02.213077 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 11:06:02.213077 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 11:06:02.213077 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 11:06:02.213077 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:06:02.213077 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:06:02.213077 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:06:02.213077 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:06:02.213077 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:06:02.213077 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:06:02.213077 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:06:02.213077 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jun 29 11:06:02.213077 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:06:02.213077 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:06:02.213077 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:06:02.213077 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:06:02.214101 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:06:02.214101 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:06:02.214101 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:06:02.214101 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:06:02.214101 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 11:06:02.214101 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 11:06:02.214101 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 11:06:02.214101 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 11:06:02.214101 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 11:06:02.214101 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 11:06:02.214101 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 29 11:06:02.214101 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jun 29 11:06:02.214101 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 101/189] 10.222.0.44 () {56 vars in 1114 bytes} [Wed Jun 29 11:06:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:06:02.214101 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f253eb3-904f-4c4d-92fb-c8944936b1ac None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:02.220099 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d837f3d7-c6be-4527-9ff7-f410e2948bbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:02.220978 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d837f3d7-c6be-4527-9ff7-f410e2948bbb None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:02.221263 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d837f3d7-c6be-4527-9ff7-f410e2948bbb None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:02.221539 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d837f3d7-c6be-4527-9ff7-f410e2948bbb None admin] PATH_INFO: `/v3/users/03fb807e4be54030b7f888a3962b0e51` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:02.235382 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d837f3d7-c6be-4527-9ff7-f410e2948bbb None admin] RBAC: Authorizing `identity:get_user(user_id=03fb807e4be54030b7f888a3962b0e51)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:02.236855 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d837f3d7-c6be-4527-9ff7-f410e2948bbb None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:02.238497 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 89/190] 10.222.0.44 () {56 vars in 1177 bytes} [Wed Jun 29 11:06:02 2022] GET /identity/v3/users/03fb807e4be54030b7f888a3962b0e51 => generated 283 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 11:06:02.239690 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b942fe-6bad-40b0-9cb7-96f865f65dbd None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:02.240016 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b942fe-6bad-40b0-9cb7-96f865f65dbd None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:02.240289 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b942fe-6bad-40b0-9cb7-96f865f65dbd None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:02.241539 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 90/191] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:06:02 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:06:02.242217 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b942fe-6bad-40b0-9cb7-96f865f65dbd None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:02.282348 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:02.321601 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f253eb3-904f-4c4d-92fb-c8944936b1ac None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:02.338705 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:02.343470 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b942fe-6bad-40b0-9cb7-96f865f65dbd None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:02.355606 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fdf9bb-15d6-433f-af9f-21cca421c26d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:02.356487 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fdf9bb-15d6-433f-af9f-21cca421c26d None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:02.356820 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fdf9bb-15d6-433f-af9f-21cca421c26d None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:02.357054 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fdf9bb-15d6-433f-af9f-21cca421c26d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:02.358228 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fdf9bb-15d6-433f-af9f-21cca421c26d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:02.359702 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fdf9bb-15d6-433f-af9f-21cca421c26d None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:02.361350 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 102/192] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jun 29 11:06:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:02.362108 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fdf9bb-15d6-433f-af9f-21cca421c26d None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:02.377779 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2165cb49-26d8-4b1a-8166-02c951ebc9e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:02.378514 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2165cb49-26d8-4b1a-8166-02c951ebc9e1 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:02.378774 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2165cb49-26d8-4b1a-8166-02c951ebc9e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:02.379045 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2165cb49-26d8-4b1a-8166-02c951ebc9e1 None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:02.379784 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2165cb49-26d8-4b1a-8166-02c951ebc9e1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:06:02.380119 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2165cb49-26d8-4b1a-8166-02c951ebc9e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:02.381555 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2165cb49-26d8-4b1a-8166-02c951ebc9e1 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:02.390257 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 91/193] 10.222.0.44 () {56 vars in 1111 bytes} [Wed Jun 29 11:06:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:02.391278 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b64e006-d89e-42e2-887f-1e2a70709590 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:02.391569 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b64e006-d89e-42e2-887f-1e2a70709590 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:02.391869 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b64e006-d89e-42e2-887f-1e2a70709590 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:02.408077 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-7b64e006-d89e-42e2-887f-1e2a70709590 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:06:02.409298 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-7b64e006-d89e-42e2-887f-1e2a70709590 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:06:02.463817 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fdf9bb-15d6-433f-af9f-21cca421c26d None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:02.478641 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 92/194] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:06:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 11:06:02.479466 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b64e006-d89e-42e2-887f-1e2a70709590 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:02.499810 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912faca1-1bbe-4670-a899-8344f6b41ed0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:02.500496 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912faca1-1bbe-4670-a899-8344f6b41ed0 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:02.500782 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912faca1-1bbe-4670-a899-8344f6b41ed0 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:02.501061 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912faca1-1bbe-4670-a899-8344f6b41ed0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:02.516174 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:02.552807 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:02.568219 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912faca1-1bbe-4670-a899-8344f6b41ed0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:02.570237 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912faca1-1bbe-4670-a899-8344f6b41ed0 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:02.587140 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b64e006-d89e-42e2-887f-1e2a70709590 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:02.621011 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581b6aa6-eb6d-4fa6-a5fa-0fa86bc13c84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:02.621607 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581b6aa6-eb6d-4fa6-a5fa-0fa86bc13c84 None admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:02.621889 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581b6aa6-eb6d-4fa6-a5fa-0fa86bc13c84 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:02.622166 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581b6aa6-eb6d-4fa6-a5fa-0fa86bc13c84 None admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:02.622870 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581b6aa6-eb6d-4fa6-a5fa-0fa86bc13c84 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:02.624341 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581b6aa6-eb6d-4fa6-a5fa-0fa86bc13c84 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:02.667248 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 103/195] 10.222.0.44 () {58 vars in 1282 bytes} [Wed Jun 29 11:06:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 29 11:06:02.668333 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912faca1-1bbe-4670-a899-8344f6b41ed0 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:02.719142 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 93/196] 10.222.0.44 () {60 vars in 1132 bytes} [Wed Jun 29 11:06:02 2022] POST /identity/v3/users => generated 297 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:06:02.736890 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581b6aa6-eb6d-4fa6-a5fa-0fa86bc13c84 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:02.754842 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:02.777313 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912faca1-1bbe-4670-a899-8344f6b41ed0 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:02.821860 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f0baa5-0a80-40c0-886c-aee581e40cc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:02.823012 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f0baa5-0a80-40c0-886c-aee581e40cc0 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:02.823324 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f0baa5-0a80-40c0-886c-aee581e40cc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:02.823602 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f0baa5-0a80-40c0-886c-aee581e40cc0 None admin] PATH_INFO: `/v3/users/03fb807e4be54030b7f888a3962b0e51` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:02.824921 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f0baa5-0a80-40c0-886c-aee581e40cc0 None admin] RBAC: Authorizing `identity:get_user(user_id=03fb807e4be54030b7f888a3962b0e51)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:02.826550 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f0baa5-0a80-40c0-886c-aee581e40cc0 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:02.829009 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 104/197] 10.222.0.44 () {56 vars in 1177 bytes} [Wed Jun 29 11:06:02 2022] GET /identity/v3/users/03fb807e4be54030b7f888a3962b0e51 => generated 283 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:02.829673 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f0baa5-0a80-40c0-886c-aee581e40cc0 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:02.848383 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:02.864824 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581b6aa6-eb6d-4fa6-a5fa-0fa86bc13c84 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:02.903169 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9ba3c8-57b3-4ec3-83e8-f4dcb43b02dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:02.904043 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9ba3c8-57b3-4ec3-83e8-f4dcb43b02dd None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:02.904253 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9ba3c8-57b3-4ec3-83e8-f4dcb43b02dd None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:02.904511 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9ba3c8-57b3-4ec3-83e8-f4dcb43b02dd None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:02.921223 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9ba3c8-57b3-4ec3-83e8-f4dcb43b02dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 11:06:02.929693 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-0b9ba3c8-57b3-4ec3-83e8-f4dcb43b02dd None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 29 11:06:02.929693 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:06:02.929693 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:06:02.929693 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:06:02.929693 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:06:02.929693 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:06:02.929693 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:06:02.929693 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:06:02.929693 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:06:02.929693 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:06:02.929693 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:06:02.929693 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:06:02.929693 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 11:06:02.929693 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 11:06:02.929693 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 11:06:02.929693 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 11:06:02.930620 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 11:06:02.930620 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 11:06:02.930620 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 11:06:02.930620 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 11:06:02.930620 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:06:02.930620 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:06:02.930620 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:06:02.930620 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:06:02.930620 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:06:02.930620 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:06:02.930620 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:06:02.930620 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jun 29 11:06:02.930620 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:06:02.930620 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:06:02.930620 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:06:02.930620 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:06:02.931843 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:06:02.931843 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:06:02.931843 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:06:02.931843 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:06:02.931843 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 11:06:02.931843 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 11:06:02.931843 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 11:06:02.931843 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 11:06:02.931843 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 11:06:02.931843 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 11:06:02.931843 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 29 11:06:02.931843 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:06:02.931843 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 94/198] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Jun 29 11:06:02 2022] GET /identity/v3/roles/reader => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:06:02.931843 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9ba3c8-57b3-4ec3-83e8-f4dcb43b02dd None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:02.949155 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f0baa5-0a80-40c0-886c-aee581e40cc0 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:02.989461 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19d2bfc-5a0d-4c95-b357-0640329ee9f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:02.990479 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19d2bfc-5a0d-4c95-b357-0640329ee9f2 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:02.990765 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19d2bfc-5a0d-4c95-b357-0640329ee9f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:02.991037 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19d2bfc-5a0d-4c95-b357-0640329ee9f2 None admin] PATH_INFO: `/v3/projects/d36c6c9eb3234e82bb6a8537851eb336` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:02.992943 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19d2bfc-5a0d-4c95-b357-0640329ee9f2 None admin] RBAC: Authorizing `identity:get_project(project_id=d36c6c9eb3234e82bb6a8537851eb336)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:02.994499 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19d2bfc-5a0d-4c95-b357-0640329ee9f2 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:02.996472 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 105/199] 10.222.0.44 () {56 vars in 1186 bytes} [Wed Jun 29 11:06:02 2022] GET /identity/v3/projects/d36c6c9eb3234e82bb6a8537851eb336 => generated 302 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:02.997239 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19d2bfc-5a0d-4c95-b357-0640329ee9f2 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:03.023723 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:03.056382 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9ba3c8-57b3-4ec3-83e8-f4dcb43b02dd None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:03.098757 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10ccf18-b76a-4846-8f51-08ecf3b87d06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:03.099698 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10ccf18-b76a-4846-8f51-08ecf3b87d06 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:03.100039 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10ccf18-b76a-4846-8f51-08ecf3b87d06 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:03.100324 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10ccf18-b76a-4846-8f51-08ecf3b87d06 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:03.120499 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:03.123228 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19d2bfc-5a0d-4c95-b357-0640329ee9f2 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:03.162311 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1216cd3a-5bc8-44b6-8af3-abb4275d0f25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:03.162901 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1216cd3a-5bc8-44b6-8af3-abb4275d0f25 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:03.163722 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1216cd3a-5bc8-44b6-8af3-abb4275d0f25 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:03.164057 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1216cd3a-5bc8-44b6-8af3-abb4275d0f25 None admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:03.164817 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1216cd3a-5bc8-44b6-8af3-abb4275d0f25 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:06:03.165126 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1216cd3a-5bc8-44b6-8af3-abb4275d0f25 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:03.166512 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1216cd3a-5bc8-44b6-8af3-abb4275d0f25 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:03.175404 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 106/200] 10.222.0.44 () {56 vars in 1101 bytes} [Wed Jun 29 11:06:02 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:03.176365 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1216cd3a-5bc8-44b6-8af3-abb4275d0f25 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:03.193885 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10ccf18-b76a-4846-8f51-08ecf3b87d06 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:03.196005 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10ccf18-b76a-4846-8f51-08ecf3b87d06 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:03.295687 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:03.322609 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 95/201] 10.222.0.44 () {58 vars in 1282 bytes} [Wed Jun 29 11:06:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 29 11:06:03.324412 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10ccf18-b76a-4846-8f51-08ecf3b87d06 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:03.329580 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1216cd3a-5bc8-44b6-8af3-abb4275d0f25 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:03.367639 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee30fa2-8f02-406a-a229-e5f9f1cbbb57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:03.368750 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee30fa2-8f02-406a-a229-e5f9f1cbbb57 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:03.375155 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee30fa2-8f02-406a-a229-e5f9f1cbbb57 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:03.375155 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee30fa2-8f02-406a-a229-e5f9f1cbbb57 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:03.375155 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee30fa2-8f02-406a-a229-e5f9f1cbbb57 None admin] RBAC: Adding query filter params (role.id=e95c2ce63e114cc19a9dda6ad21e794f, scope.project.id=d36c6c9eb3234e82bb6a8537851eb336, user.id=03fb807e4be54030b7f888a3962b0e51) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:06:03.375155 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee30fa2-8f02-406a-a229-e5f9f1cbbb57 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:03.375155 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee30fa2-8f02-406a-a229-e5f9f1cbbb57 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:03.383975 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 107/202] 10.222.0.44 () {56 vars in 1374 bytes} [Wed Jun 29 11:06:03 2022] GET /identity/v3/role_assignments?user.id=03fb807e4be54030b7f888a3962b0e51&scope.project.id=d36c6c9eb3234e82bb6a8537851eb336&role.id=e95c2ce63e114cc19a9dda6ad21e794f => generated 259 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:03.384591 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:03.386451 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee30fa2-8f02-406a-a229-e5f9f1cbbb57 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:03.512222 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10ccf18-b76a-4846-8f51-08ecf3b87d06 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:03.550273 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee30fa2-8f02-406a-a229-e5f9f1cbbb57 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:03.579309 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5297227e-ef05-495c-9b59-06bd1b9e477b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:03.580496 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5297227e-ef05-495c-9b59-06bd1b9e477b None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:03.580970 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5297227e-ef05-495c-9b59-06bd1b9e477b None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:03.581402 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5297227e-ef05-495c-9b59-06bd1b9e477b None admin] PATH_INFO: `/v3/users/c444598ebd8845d2a9cdf41cefa70e01` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:03.582780 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5297227e-ef05-495c-9b59-06bd1b9e477b None admin] RBAC: Authorizing `identity:get_user(user_id=c444598ebd8845d2a9cdf41cefa70e01)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:03.584421 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5297227e-ef05-495c-9b59-06bd1b9e477b None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:03.586688 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 96/203] 10.222.0.44 () {56 vars in 1177 bytes} [Wed Jun 29 11:06:03 2022] GET /identity/v3/users/c444598ebd8845d2a9cdf41cefa70e01 => generated 289 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:03.592716 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5297227e-ef05-495c-9b59-06bd1b9e477b None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:03.598452 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5902435b-4621-4ee5-81ec-a8c4093f4e28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:03.599663 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5902435b-4621-4ee5-81ec-a8c4093f4e28 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:03.601923 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5902435b-4621-4ee5-81ec-a8c4093f4e28 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:03.602381 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5902435b-4621-4ee5-81ec-a8c4093f4e28 None admin] PATH_INFO: `/v3/projects/d36c6c9eb3234e82bb6a8537851eb336` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:03.609983 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5902435b-4621-4ee5-81ec-a8c4093f4e28 None admin] RBAC: Authorizing `identity:get_project(project_id=d36c6c9eb3234e82bb6a8537851eb336)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:03.611705 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5902435b-4621-4ee5-81ec-a8c4093f4e28 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:03.613700 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 108/204] 10.222.0.44 () {56 vars in 1186 bytes} [Wed Jun 29 11:06:03 2022] GET /identity/v3/projects/d36c6c9eb3234e82bb6a8537851eb336 => generated 302 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:03.620442 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5902435b-4621-4ee5-81ec-a8c4093f4e28 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:03.640350 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:03.739937 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5297227e-ef05-495c-9b59-06bd1b9e477b None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:03.754977 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:03.778928 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2407ab-c982-4c1d-875d-66547bf3b76b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:03.779961 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2407ab-c982-4c1d-875d-66547bf3b76b None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:03.779961 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2407ab-c982-4c1d-875d-66547bf3b76b None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:03.780140 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2407ab-c982-4c1d-875d-66547bf3b76b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:03.829786 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5902435b-4621-4ee5-81ec-a8c4093f4e28 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:03.857248 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2407ab-c982-4c1d-875d-66547bf3b76b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:03.858721 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2407ab-c982-4c1d-875d-66547bf3b76b None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:03.964582 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b480c0-0f90-4ebb-a72c-9662f81f35da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:03.965253 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b480c0-0f90-4ebb-a72c-9662f81f35da None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:03.969357 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b480c0-0f90-4ebb-a72c-9662f81f35da None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:03.969673 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b480c0-0f90-4ebb-a72c-9662f81f35da None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:03.970555 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b480c0-0f90-4ebb-a72c-9662f81f35da None admin] RBAC: Adding query filter params (role.id=fc91ce4ad4a44cc3b05d0936754e87ec, scope.project.id=d36c6c9eb3234e82bb6a8537851eb336, user.id=03fb807e4be54030b7f888a3962b0e51) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:06:03.970872 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b480c0-0f90-4ebb-a72c-9662f81f35da None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:03.972384 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b480c0-0f90-4ebb-a72c-9662f81f35da None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:03.975853 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 97/205] 10.222.0.44 () {58 vars in 1282 bytes} [Wed Jun 29 11:06:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 29 11:06:03.982131 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2407ab-c982-4c1d-875d-66547bf3b76b None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:03.987153 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 109/206] 10.222.0.44 () {56 vars in 1374 bytes} [Wed Jun 29 11:06:03 2022] GET /identity/v3/role_assignments?user.id=03fb807e4be54030b7f888a3962b0e51&scope.project.id=d36c6c9eb3234e82bb6a8537851eb336&role.id=fc91ce4ad4a44cc3b05d0936754e87ec => generated 259 bytes in 367 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 11:06:04.014213 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:04.158027 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2407ab-c982-4c1d-875d-66547bf3b76b None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:04.210889 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a71a49-b374-4a31-b974-1c0490afbce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:04.212158 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a71a49-b374-4a31-b974-1c0490afbce1 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:04.212645 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a71a49-b374-4a31-b974-1c0490afbce1 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:04.213120 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a71a49-b374-4a31-b974-1c0490afbce1 None admin] PATH_INFO: `/v3/projects/3b98d22914de4e3b8bca7fe0a0b51357` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:04.214549 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a71a49-b374-4a31-b974-1c0490afbce1 None admin] RBAC: Authorizing `identity:get_project(project_id=3b98d22914de4e3b8bca7fe0a0b51357)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:04.216353 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a71a49-b374-4a31-b974-1c0490afbce1 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:04.218588 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 98/207] 10.222.0.44 () {56 vars in 1186 bytes} [Wed Jun 29 11:06:03 2022] GET /identity/v3/projects/3b98d22914de4e3b8bca7fe0a0b51357 => generated 298 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:04.224814 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b480c0-0f90-4ebb-a72c-9662f81f35da None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:04.323486 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:04.351344 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b480c0-0f90-4ebb-a72c-9662f81f35da None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:04.396085 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bc93ad-8405-408a-90db-f888628e2c4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:04.397101 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bc93ad-8405-408a-90db-f888628e2c4e None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:04.397616 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bc93ad-8405-408a-90db-f888628e2c4e None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:04.398090 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bc93ad-8405-408a-90db-f888628e2c4e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:04.399126 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bc93ad-8405-408a-90db-f888628e2c4e None admin] RBAC: Adding query filter params (role.id=7416219b213f48a4a93d964e67f990fe, scope.project.id=3b98d22914de4e3b8bca7fe0a0b51357, user.id=c444598ebd8845d2a9cdf41cefa70e01) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:06:04.399655 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bc93ad-8405-408a-90db-f888628e2c4e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:04.401345 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bc93ad-8405-408a-90db-f888628e2c4e None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:04.408246 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 110/208] 10.222.0.44 () {56 vars in 1374 bytes} [Wed Jun 29 11:06:04 2022] GET /identity/v3/role_assignments?user.id=c444598ebd8845d2a9cdf41cefa70e01&scope.project.id=3b98d22914de4e3b8bca7fe0a0b51357&role.id=7416219b213f48a4a93d964e67f990fe => generated 608 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:05.111574 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c2e87b-6415-4f90-ad48-e59cfb679bcb None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:05.112134 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c2e87b-6415-4f90-ad48-e59cfb679bcb None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:05.112134 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c2e87b-6415-4f90-ad48-e59cfb679bcb None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:05.113523 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 99/209] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:06:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:06:05.119452 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde4eb11-dec3-44e3-bff2-b8518b1d4f02 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:05.119876 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde4eb11-dec3-44e3-bff2-b8518b1d4f02 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:05.120052 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde4eb11-dec3-44e3-bff2-b8518b1d4f02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:05.139681 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-fde4eb11-dec3-44e3-bff2-b8518b1d4f02 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:06:05.140897 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-fde4eb11-dec3-44e3-bff2-b8518b1d4f02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:06:05.207768 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 111/210] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:06:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 11:06:05.232072 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c2e87b-6415-4f90-ad48-e59cfb679bcb None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:05.329371 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:05.349497 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c2e87b-6415-4f90-ad48-e59cfb679bcb None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:05.384440 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1ef993-cdee-4483-9ce0-d6f7e31404d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:05.385557 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1ef993-cdee-4483-9ce0-d6f7e31404d7 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:05.386009 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1ef993-cdee-4483-9ce0-d6f7e31404d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:05.386452 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1ef993-cdee-4483-9ce0-d6f7e31404d7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:05.396841 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1ef993-cdee-4483-9ce0-d6f7e31404d7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 11:06:05.405209 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-2d1ef993-cdee-4483-9ce0-d6f7e31404d7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 29 11:06:05.405209 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:06:05.405209 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:06:05.405209 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:06:05.405209 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:06:05.405209 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:06:05.405209 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:06:05.405209 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:06:05.405209 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:06:05.405209 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:06:05.405209 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:06:05.405209 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:06:05.405209 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 11:06:05.405209 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 11:06:05.405209 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 11:06:05.405209 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 11:06:05.412368 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 11:06:05.412368 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 11:06:05.412368 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 11:06:05.412368 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 11:06:05.412368 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:06:05.412368 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:06:05.412368 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:06:05.412368 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:06:05.412368 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:06:05.412368 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:06:05.412368 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:06:05.412368 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jun 29 11:06:05.412368 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:06:05.412368 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:06:05.412368 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:06:05.412368 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:06:05.413712 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:06:05.413712 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:06:05.413712 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:06:05.413712 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:06:05.413712 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 11:06:05.413712 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 11:06:05.413712 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 11:06:05.413712 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 11:06:05.413712 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 11:06:05.413712 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 11:06:05.413712 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 29 11:06:05.413712 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:06:05.413712 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 100/211] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Jun 29 11:06:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:06:05.416937 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde4eb11-dec3-44e3-bff2-b8518b1d4f02 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:05.421537 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb6a67c-b71d-462a-ae20-f3aafbbe7f1e None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:05.422014 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb6a67c-b71d-462a-ae20-f3aafbbe7f1e None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:05.422488 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb6a67c-b71d-462a-ae20-f3aafbbe7f1e None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:05.424049 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 101/212] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:06:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:06:05.430865 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c536cd69-0302-4e68-a918-939d33af96f0 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:05.431335 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c536cd69-0302-4e68-a918-939d33af96f0 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:05.431778 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c536cd69-0302-4e68-a918-939d33af96f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:05.449022 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-c536cd69-0302-4e68-a918-939d33af96f0 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:06:05.450446 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-c536cd69-0302-4e68-a918-939d33af96f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:06:05.496644 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:05.528693 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 102/213] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:06:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 11:06:05.531947 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde4eb11-dec3-44e3-bff2-b8518b1d4f02 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:05.561368 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c536cd69-0302-4e68-a918-939d33af96f0 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:05.580369 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e60ff5-4aa5-41a1-9f45-d9608c9fe1be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:05.581054 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e60ff5-4aa5-41a1-9f45-d9608c9fe1be None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:05.581339 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e60ff5-4aa5-41a1-9f45-d9608c9fe1be None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:05.581638 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e60ff5-4aa5-41a1-9f45-d9608c9fe1be None admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:05.582486 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e60ff5-4aa5-41a1-9f45-d9608c9fe1be None admin] RBAC: Adding query filter params (name=member) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:06:05.582799 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e60ff5-4aa5-41a1-9f45-d9608c9fe1be None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:05.584334 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e60ff5-4aa5-41a1-9f45-d9608c9fe1be None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:05.598670 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 112/214] 10.222.0.44 () {56 vars in 1101 bytes} [Wed Jun 29 11:06:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:05.601143 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e60ff5-4aa5-41a1-9f45-d9608c9fe1be None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:05.653491 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:05.695737 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c536cd69-0302-4e68-a918-939d33af96f0 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:05.731120 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e60ff5-4aa5-41a1-9f45-d9608c9fe1be None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:05.731616 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fce0e4-3d55-4c28-a991-c5a80054ad5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:05.732745 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fce0e4-3d55-4c28-a991-c5a80054ad5e None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:05.733175 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fce0e4-3d55-4c28-a991-c5a80054ad5e None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:05.733617 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fce0e4-3d55-4c28-a991-c5a80054ad5e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:05.734935 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fce0e4-3d55-4c28-a991-c5a80054ad5e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:05.736547 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fce0e4-3d55-4c28-a991-c5a80054ad5e None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:05.743378 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 103/215] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jun 29 11:06:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:05.744384 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fce0e4-3d55-4c28-a991-c5a80054ad5e None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:05.771283 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597cbd1b-c1a1-45ba-a817-16e8d28912dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:05.772344 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597cbd1b-c1a1-45ba-a817-16e8d28912dd None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:05.772732 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597cbd1b-c1a1-45ba-a817-16e8d28912dd None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:05.773110 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597cbd1b-c1a1-45ba-a817-16e8d28912dd None admin] PATH_INFO: `/v3/users/bfeaeb065ed045e7b9d3e24a65869673` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:05.775477 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:05.789555 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597cbd1b-c1a1-45ba-a817-16e8d28912dd None admin] RBAC: Authorizing `identity:get_user(user_id=bfeaeb065ed045e7b9d3e24a65869673)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:05.791399 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597cbd1b-c1a1-45ba-a817-16e8d28912dd None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:05.793651 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 113/216] 10.222.0.44 () {56 vars in 1177 bytes} [Wed Jun 29 11:06:05 2022] GET /identity/v3/users/bfeaeb065ed045e7b9d3e24a65869673 => generated 297 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:05.799169 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597cbd1b-c1a1-45ba-a817-16e8d28912dd None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:05.852002 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fce0e4-3d55-4c28-a991-c5a80054ad5e None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:05.887108 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4115af6-e876-4840-9599-97791ddd5412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:05.888051 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4115af6-e876-4840-9599-97791ddd5412 None admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:05.888497 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4115af6-e876-4840-9599-97791ddd5412 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:05.888955 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4115af6-e876-4840-9599-97791ddd5412 None admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:05.889843 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4115af6-e876-4840-9599-97791ddd5412 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:05.891490 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4115af6-e876-4840-9599-97791ddd5412 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:05.902231 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:05.923933 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597cbd1b-c1a1-45ba-a817-16e8d28912dd None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:05.978611 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cff04e-8215-4c16-97fe-e1f185f5a5c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:05.979267 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cff04e-8215-4c16-97fe-e1f185f5a5c3 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:05.979551 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cff04e-8215-4c16-97fe-e1f185f5a5c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:05.979847 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cff04e-8215-4c16-97fe-e1f185f5a5c3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:06.031166 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 104/217] 10.222.0.44 () {60 vars in 1132 bytes} [Wed Jun 29 11:06:05 2022] POST /identity/v3/users => generated 297 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:06:06.031166 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4636fd98-5808-4707-bab6-8962fbce2c3b None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:06.031166 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4636fd98-5808-4707-bab6-8962fbce2c3b None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:06.031166 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4636fd98-5808-4707-bab6-8962fbce2c3b None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:06.031614 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 105/218] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:06:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:06:06.037574 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c98804-d980-414f-972c-7ec1cd2bd6c7 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:06.038576 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c98804-d980-414f-972c-7ec1cd2bd6c7 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:06.038931 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c98804-d980-414f-972c-7ec1cd2bd6c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:06.049366 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:06.061505 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-50c98804-d980-414f-972c-7ec1cd2bd6c7 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:06:06.062391 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cff04e-8215-4c16-97fe-e1f185f5a5c3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:06.063293 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-50c98804-d980-414f-972c-7ec1cd2bd6c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:06:06.064033 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cff04e-8215-4c16-97fe-e1f185f5a5c3 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:06.142519 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 106/219] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:06:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 29 11:06:06.167882 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c98804-d980-414f-972c-7ec1cd2bd6c7 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:06.210589 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 114/220] 10.222.0.44 () {58 vars in 1282 bytes} [Wed Jun 29 11:06:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 29 11:06:06.211618 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cff04e-8215-4c16-97fe-e1f185f5a5c3 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:06.299891 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c98804-d980-414f-972c-7ec1cd2bd6c7 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:06.344176 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:06.346455 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:06.353796 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cff04e-8215-4c16-97fe-e1f185f5a5c3 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:06.358248 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2332e730-f1dc-4111-9d7c-3c7a6afa8d95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:06.359586 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2332e730-f1dc-4111-9d7c-3c7a6afa8d95 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:06.360027 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2332e730-f1dc-4111-9d7c-3c7a6afa8d95 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:06.360502 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2332e730-f1dc-4111-9d7c-3c7a6afa8d95 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:06.373210 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2332e730-f1dc-4111-9d7c-3c7a6afa8d95 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 11:06:06.381711 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-2332e730-f1dc-4111-9d7c-3c7a6afa8d95 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 11:06:06.381711 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:06:06.381711 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:06:06.381711 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:06:06.381711 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:06:06.381711 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:06:06.381711 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:06:06.381711 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:06:06.381711 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:06:06.381711 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:06:06.381711 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:06:06.381711 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:06:06.381711 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 11:06:06.381711 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 11:06:06.381711 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 11:06:06.381711 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 11:06:06.382622 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 11:06:06.382622 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 11:06:06.382622 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 11:06:06.382622 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 11:06:06.382622 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:06:06.382622 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:06:06.382622 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:06:06.382622 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:06:06.382622 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:06:06.382622 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:06:06.382622 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:06:06.382622 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jun 29 11:06:06.382622 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:06:06.382622 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:06:06.382622 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:06:06.382622 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:06:06.383419 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:06:06.383419 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:06:06.383419 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:06:06.383419 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:06:06.383419 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 11:06:06.383419 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 11:06:06.383419 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 11:06:06.383419 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 11:06:06.383419 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 11:06:06.383419 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 11:06:06.383419 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 11:06:06.383419 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:06:06.383419 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 107/221] 10.222.0.44 () {56 vars in 1096 bytes} [Wed Jun 29 11:06:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 11:06:06.385774 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a846666-dcba-4118-8692-178f3596b8b5 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:06.385774 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a846666-dcba-4118-8692-178f3596b8b5 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:06.385774 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a846666-dcba-4118-8692-178f3596b8b5 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:06.386432 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 108/222] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:06:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:06:06.392139 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df90edba-0aa5-4f3a-ab7a-23798ed0afa9 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:06.392429 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df90edba-0aa5-4f3a-ab7a-23798ed0afa9 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:06.392710 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df90edba-0aa5-4f3a-ab7a-23798ed0afa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:06.404797 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01797cb-c0be-44f2-95b7-6b30737c637f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:06.405721 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01797cb-c0be-44f2-95b7-6b30737c637f None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:06.406004 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01797cb-c0be-44f2-95b7-6b30737c637f None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:06.406275 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01797cb-c0be-44f2-95b7-6b30737c637f None admin] PATH_INFO: `/v3/projects/d36c6c9eb3234e82bb6a8537851eb336` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:06.414611 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01797cb-c0be-44f2-95b7-6b30737c637f None admin] RBAC: Authorizing `identity:get_project(project_id=d36c6c9eb3234e82bb6a8537851eb336)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:06.414611 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01797cb-c0be-44f2-95b7-6b30737c637f None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:06.414611 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 115/223] 10.222.0.44 () {56 vars in 1186 bytes} [Wed Jun 29 11:06:06 2022] GET /identity/v3/projects/d36c6c9eb3234e82bb6a8537851eb336 => generated 302 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:06.414868 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-df90edba-0aa5-4f3a-ab7a-23798ed0afa9 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:06:06.414940 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01797cb-c0be-44f2-95b7-6b30737c637f None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:06.415014 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-df90edba-0aa5-4f3a-ab7a-23798ed0afa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:06:06.512921 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 109/224] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:06:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 29 11:06:06.515322 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df90edba-0aa5-4f3a-ab7a-23798ed0afa9 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:06.550863 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01797cb-c0be-44f2-95b7-6b30737c637f None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:06.588478 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce1050d-97f2-41a5-8065-7ec6ef52288f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:06.589531 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce1050d-97f2-41a5-8065-7ec6ef52288f None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:06.589985 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce1050d-97f2-41a5-8065-7ec6ef52288f None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:06.590458 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce1050d-97f2-41a5-8065-7ec6ef52288f None admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:06.591377 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce1050d-97f2-41a5-8065-7ec6ef52288f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:06:06.591871 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce1050d-97f2-41a5-8065-7ec6ef52288f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:06.593498 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce1050d-97f2-41a5-8065-7ec6ef52288f None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:06.608414 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 116/225] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Jun 29 11:06:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:06.609540 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce1050d-97f2-41a5-8065-7ec6ef52288f None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:06.633859 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df90edba-0aa5-4f3a-ab7a-23798ed0afa9 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:06.636714 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:06.651760 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:06.673883 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16f7aa6-331e-4b24-a893-940225e3af8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:06.675116 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16f7aa6-331e-4b24-a893-940225e3af8c None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:06.675799 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16f7aa6-331e-4b24-a893-940225e3af8c None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:06.676651 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16f7aa6-331e-4b24-a893-940225e3af8c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:06.678060 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16f7aa6-331e-4b24-a893-940225e3af8c None admin] RBAC: Adding query filter params (role.id=e1c88eebe911432cab6a07fbeb84653b, scope.project.id=d36c6c9eb3234e82bb6a8537851eb336, user.id=bfeaeb065ed045e7b9d3e24a65869673) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:06:06.679543 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16f7aa6-331e-4b24-a893-940225e3af8c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:06.681976 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16f7aa6-331e-4b24-a893-940225e3af8c None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:06.689453 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 110/226] 10.222.0.44 () {56 vars in 1374 bytes} [Wed Jun 29 11:06:06 2022] GET /identity/v3/role_assignments?user.id=bfeaeb065ed045e7b9d3e24a65869673&scope.project.id=d36c6c9eb3234e82bb6a8537851eb336&role.id=e1c88eebe911432cab6a07fbeb84653b => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:06.690626 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16f7aa6-331e-4b24-a893-940225e3af8c None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:06.754855 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce1050d-97f2-41a5-8065-7ec6ef52288f None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:06.794579 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9774f96e-556f-4689-98a2-0aeaebbbc9a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:06.795971 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9774f96e-556f-4689-98a2-0aeaebbbc9a0 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:06.796455 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9774f96e-556f-4689-98a2-0aeaebbbc9a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:06.796924 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9774f96e-556f-4689-98a2-0aeaebbbc9a0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:06.809432 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9774f96e-556f-4689-98a2-0aeaebbbc9a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 11:06:06.818562 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-9774f96e-556f-4689-98a2-0aeaebbbc9a0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 29 11:06:06.818562 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:06:06.818562 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:06:06.818562 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:06:06.818562 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:06:06.818562 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:06:06.818562 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:06:06.818562 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:06:06.818562 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:06:06.818562 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:06:06.818562 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:06:06.818562 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:06:06.818562 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 11:06:06.818562 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 11:06:06.818562 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 11:06:06.818562 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 11:06:06.820025 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 11:06:06.820025 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 11:06:06.820025 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 11:06:06.820025 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 11:06:06.820025 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:06:06.820025 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:06:06.820025 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:06:06.820025 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:06:06.820025 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:06:06.820025 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:06:06.820025 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:06:06.820025 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jun 29 11:06:06.820025 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:06:06.820025 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:06:06.820025 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:06:06.820025 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:06:06.820857 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:06:06.820857 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:06:06.820857 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:06:06.820857 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:06:06.820857 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 11:06:06.820857 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 11:06:06.820857 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 11:06:06.820857 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 11:06:06.820857 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 11:06:06.820857 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 11:06:06.820857 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 29 11:06:06.820857 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jun 29 11:06:06.824009 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 117/227] 10.222.0.44 () {56 vars in 1114 bytes} [Wed Jun 29 11:06:06 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:06:06.830227 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9774f96e-556f-4689-98a2-0aeaebbbc9a0 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:06.863881 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16f7aa6-331e-4b24-a893-940225e3af8c None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:06.904247 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98085cc-b19c-4d50-9b76-11be71e2fe53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:06.905147 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98085cc-b19c-4d50-9b76-11be71e2fe53 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:06.905428 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98085cc-b19c-4d50-9b76-11be71e2fe53 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:06.905704 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98085cc-b19c-4d50-9b76-11be71e2fe53 None admin] PATH_INFO: `/v3/users/03fb807e4be54030b7f888a3962b0e51` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:06.907072 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98085cc-b19c-4d50-9b76-11be71e2fe53 None admin] RBAC: Authorizing `identity:get_user(user_id=03fb807e4be54030b7f888a3962b0e51)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:06.908604 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98085cc-b19c-4d50-9b76-11be71e2fe53 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:06.910333 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 111/228] 10.222.0.44 () {56 vars in 1177 bytes} [Wed Jun 29 11:06:06 2022] GET /identity/v3/users/03fb807e4be54030b7f888a3962b0e51 => generated 283 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:06.917736 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98085cc-b19c-4d50-9b76-11be71e2fe53 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:06.993334 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:07.002697 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:07.014603 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9774f96e-556f-4689-98a2-0aeaebbbc9a0 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:07.063349 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98085cc-b19c-4d50-9b76-11be71e2fe53 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:07.101236 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0caa5e4-a4c7-4b5c-a0ce-9fe06dafae61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:07.101432 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0bded6-8c59-42ac-abb7-852787c17dc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:07.101917 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0caa5e4-a4c7-4b5c-a0ce-9fe06dafae61 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:07.102204 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0caa5e4-a4c7-4b5c-a0ce-9fe06dafae61 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:07.102465 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0bded6-8c59-42ac-abb7-852787c17dc0 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:07.102566 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0caa5e4-a4c7-4b5c-a0ce-9fe06dafae61 None admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:07.102818 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0bded6-8c59-42ac-abb7-852787c17dc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:07.103169 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0bded6-8c59-42ac-abb7-852787c17dc0 None admin] PATH_INFO: `/v3/projects/d36c6c9eb3234e82bb6a8537851eb336` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:07.103265 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0caa5e4-a4c7-4b5c-a0ce-9fe06dafae61 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:06:07.103521 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0caa5e4-a4c7-4b5c-a0ce-9fe06dafae61 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:07.104522 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0bded6-8c59-42ac-abb7-852787c17dc0 None admin] RBAC: Authorizing `identity:get_project(project_id=d36c6c9eb3234e82bb6a8537851eb336)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:07.104935 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0caa5e4-a4c7-4b5c-a0ce-9fe06dafae61 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:07.106084 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0bded6-8c59-42ac-abb7-852787c17dc0 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:07.108139 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 112/229] 10.222.0.44 () {56 vars in 1186 bytes} [Wed Jun 29 11:06:06 2022] GET /identity/v3/projects/d36c6c9eb3234e82bb6a8537851eb336 => generated 302 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:07.113943 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0bded6-8c59-42ac-abb7-852787c17dc0 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:07.125750 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 118/230] 10.222.0.44 () {56 vars in 1111 bytes} [Wed Jun 29 11:06:06 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 11:06:07.143988 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0caa5e4-a4c7-4b5c-a0ce-9fe06dafae61 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:07.230911 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0bded6-8c59-42ac-abb7-852787c17dc0 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:07.250445 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:07.271276 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e4b8b9-b5ee-4c3c-883f-38a5e4ed0151 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:07.272761 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e4b8b9-b5ee-4c3c-883f-38a5e4ed0151 None admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:07.273044 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e4b8b9-b5ee-4c3c-883f-38a5e4ed0151 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:07.277580 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e4b8b9-b5ee-4c3c-883f-38a5e4ed0151 None admin] PATH_INFO: `/v3/projects/d36c6c9eb3234e82bb6a8537851eb336/users/03fb807e4be54030b7f888a3962b0e51/roles/e95c2ce63e114cc19a9dda6ad21e794f` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:07.277580 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e4b8b9-b5ee-4c3c-883f-38a5e4ed0151 None admin] RBAC: Authorizing `identity:create_grant(project_id=d36c6c9eb3234e82bb6a8537851eb336, user_id=03fb807e4be54030b7f888a3962b0e51, role_id=e95c2ce63e114cc19a9dda6ad21e794f)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:07.277580 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e4b8b9-b5ee-4c3c-883f-38a5e4ed0151 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:07.338849 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0caa5e4-a4c7-4b5c-a0ce-9fe06dafae61 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:07.369905 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 113/231] 10.222.0.44 () {58 vars in 1439 bytes} [Wed Jun 29 11:06:07 2022] PUT /identity/v3/projects/d36c6c9eb3234e82bb6a8537851eb336/users/03fb807e4be54030b7f888a3962b0e51/roles/e95c2ce63e114cc19a9dda6ad21e794f => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:06:07.378571 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecb758e-d8e3-46a0-833f-de2258fee563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:07.379487 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecb758e-d8e3-46a0-833f-de2258fee563 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:07.379773 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecb758e-d8e3-46a0-833f-de2258fee563 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:07.380082 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecb758e-d8e3-46a0-833f-de2258fee563 None admin] PATH_INFO: `/v3/users/03fb807e4be54030b7f888a3962b0e51` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:07.381530 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecb758e-d8e3-46a0-833f-de2258fee563 None admin] RBAC: Authorizing `identity:get_user(user_id=03fb807e4be54030b7f888a3962b0e51)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:07.387539 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecb758e-d8e3-46a0-833f-de2258fee563 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:07.387539 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 119/232] 10.222.0.44 () {56 vars in 1177 bytes} [Wed Jun 29 11:06:07 2022] GET /identity/v3/users/03fb807e4be54030b7f888a3962b0e51 => generated 283 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 11:06:07.390756 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e4b8b9-b5ee-4c3c-883f-38a5e4ed0151 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:07.538975 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e4b8b9-b5ee-4c3c-883f-38a5e4ed0151 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:07.578600 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef8e4af-8386-4aa1-9e63-8cb2a1e2d18a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:07.579474 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef8e4af-8386-4aa1-9e63-8cb2a1e2d18a None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:07.579751 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef8e4af-8386-4aa1-9e63-8cb2a1e2d18a None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:07.580054 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef8e4af-8386-4aa1-9e63-8cb2a1e2d18a None admin] PATH_INFO: `/v3/projects/d36c6c9eb3234e82bb6a8537851eb336` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:07.581280 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef8e4af-8386-4aa1-9e63-8cb2a1e2d18a None admin] RBAC: Authorizing `identity:get_project(project_id=d36c6c9eb3234e82bb6a8537851eb336)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:07.582728 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef8e4af-8386-4aa1-9e63-8cb2a1e2d18a None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:07.585418 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 114/233] 10.222.0.44 () {56 vars in 1186 bytes} [Wed Jun 29 11:06:07 2022] GET /identity/v3/projects/d36c6c9eb3234e82bb6a8537851eb336 => generated 302 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:07.591197 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecb758e-d8e3-46a0-833f-de2258fee563 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:07.609634 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:07.708612 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecb758e-d8e3-46a0-833f-de2258fee563 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:07.744542 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac3887e-4ff0-4499-ba9c-d6a6190ac57b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:07.745433 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac3887e-4ff0-4499-ba9c-d6a6190ac57b None admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:07.745715 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac3887e-4ff0-4499-ba9c-d6a6190ac57b None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:07.745992 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac3887e-4ff0-4499-ba9c-d6a6190ac57b None admin] PATH_INFO: `/v3/projects/d36c6c9eb3234e82bb6a8537851eb336/users/03fb807e4be54030b7f888a3962b0e51/roles/fc91ce4ad4a44cc3b05d0936754e87ec` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:07.748260 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac3887e-4ff0-4499-ba9c-d6a6190ac57b None admin] RBAC: Authorizing `identity:create_grant(project_id=d36c6c9eb3234e82bb6a8537851eb336, user_id=03fb807e4be54030b7f888a3962b0e51, role_id=fc91ce4ad4a44cc3b05d0936754e87ec)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:07.749704 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac3887e-4ff0-4499-ba9c-d6a6190ac57b None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:07.845570 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 120/234] 10.222.0.44 () {58 vars in 1439 bytes} [Wed Jun 29 11:06:07 2022] PUT /identity/v3/projects/d36c6c9eb3234e82bb6a8537851eb336/users/03fb807e4be54030b7f888a3962b0e51/roles/fc91ce4ad4a44cc3b05d0936754e87ec => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:06:08.352202 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc75524-281a-42cc-9f1f-9e2017a24809 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:08.352713 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc75524-281a-42cc-9f1f-9e2017a24809 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:08.352788 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc75524-281a-42cc-9f1f-9e2017a24809 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:08.354114 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 115/235] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:06:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:06:08.359464 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdc9fda-b585-4688-b61c-8002f9bae6ce None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:08.359875 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdc9fda-b585-4688-b61c-8002f9bae6ce None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:08.360062 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdc9fda-b585-4688-b61c-8002f9bae6ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:08.382305 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-bbdc9fda-b585-4688-b61c-8002f9bae6ce None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:06:08.383585 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-bbdc9fda-b585-4688-b61c-8002f9bae6ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:06:08.477236 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 121/236] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:06:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 11:06:08.517387 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc75524-281a-42cc-9f1f-9e2017a24809 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:08.588239 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:08.605774 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69469f17-9e57-42b5-997a-8b189278338b None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:08.606366 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69469f17-9e57-42b5-997a-8b189278338b None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:08.606857 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69469f17-9e57-42b5-997a-8b189278338b None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:08.608495 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 122/237] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:06:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:06:08.613662 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7e1c17-c69a-4131-8ff5-915b5fe11805 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:08.614181 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7e1c17-c69a-4131-8ff5-915b5fe11805 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:08.614656 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7e1c17-c69a-4131-8ff5-915b5fe11805 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:08.620509 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:08.637786 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-3e7e1c17-c69a-4131-8ff5-915b5fe11805 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:06:08.639476 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-3e7e1c17-c69a-4131-8ff5-915b5fe11805 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:06:08.692530 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc75524-281a-42cc-9f1f-9e2017a24809 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:08.718481 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 123/238] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:06:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 11:06:08.748263 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7e1c17-c69a-4131-8ff5-915b5fe11805 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:08.760789 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c797050-4dcc-40f5-a36a-12c3c5b1bbcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:08.762021 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c797050-4dcc-40f5-a36a-12c3c5b1bbcb None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:08.762503 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c797050-4dcc-40f5-a36a-12c3c5b1bbcb None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:08.763106 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c797050-4dcc-40f5-a36a-12c3c5b1bbcb None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:08.775256 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c797050-4dcc-40f5-a36a-12c3c5b1bbcb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 11:06:08.783761 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-0c797050-4dcc-40f5-a36a-12c3c5b1bbcb None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 29 11:06:08.783761 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:06:08.783761 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:06:08.783761 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:06:08.783761 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:06:08.783761 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:06:08.783761 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:06:08.783761 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:06:08.783761 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:06:08.783761 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:06:08.783761 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:06:08.783761 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:06:08.783761 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 11:06:08.783761 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 11:06:08.783761 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 11:06:08.783761 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 11:06:08.784661 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 11:06:08.784661 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 11:06:08.784661 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 11:06:08.784661 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 11:06:08.784661 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:06:08.784661 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:06:08.784661 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:06:08.784661 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:06:08.784661 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:06:08.784661 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:06:08.784661 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:06:08.784661 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jun 29 11:06:08.784661 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:06:08.784661 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:06:08.784661 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:06:08.784661 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:06:08.785470 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:06:08.785470 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:06:08.785470 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:06:08.785470 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:06:08.785470 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 11:06:08.785470 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 11:06:08.785470 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 11:06:08.785470 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 11:06:08.785470 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 11:06:08.785470 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 11:06:08.785470 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 29 11:06:08.785470 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:06:08.788430 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 116/239] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Jun 29 11:06:08 2022] GET /identity/v3/roles/reader => generated 84 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:06:08.799849 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c797050-4dcc-40f5-a36a-12c3c5b1bbcb None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:08.890254 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7e1c17-c69a-4131-8ff5-915b5fe11805 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:08.925081 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5528f1f9-1853-4fcf-86e1-a7a5ebe17e33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:08.925885 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5528f1f9-1853-4fcf-86e1-a7a5ebe17e33 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:08.926173 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5528f1f9-1853-4fcf-86e1-a7a5ebe17e33 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:08.926453 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5528f1f9-1853-4fcf-86e1-a7a5ebe17e33 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:08.927650 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5528f1f9-1853-4fcf-86e1-a7a5ebe17e33 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:08.929296 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:08.932371 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5528f1f9-1853-4fcf-86e1-a7a5ebe17e33 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:08.934330 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 124/240] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jun 29 11:06:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:08.939890 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5528f1f9-1853-4fcf-86e1-a7a5ebe17e33 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:09.014417 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c797050-4dcc-40f5-a36a-12c3c5b1bbcb None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:09.024130 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:09.051968 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda12a38-5735-4c05-9789-a26bb7944b4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:09.052804 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda12a38-5735-4c05-9789-a26bb7944b4a None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:09.053195 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda12a38-5735-4c05-9789-a26bb7944b4a None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:09.054136 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda12a38-5735-4c05-9789-a26bb7944b4a None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:09.054993 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda12a38-5735-4c05-9789-a26bb7944b4a None admin] RBAC: Adding query filter params (name=reader) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:06:09.055402 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda12a38-5735-4c05-9789-a26bb7944b4a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:09.056954 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda12a38-5735-4c05-9789-a26bb7944b4a None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:09.066663 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 117/241] 10.222.0.44 () {56 vars in 1101 bytes} [Wed Jun 29 11:06:08 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:09.071790 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda12a38-5735-4c05-9789-a26bb7944b4a None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:09.138414 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5528f1f9-1853-4fcf-86e1-a7a5ebe17e33 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:09.177933 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbdc6c4-ea30-45d3-b5c4-9180af125c50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:09.179121 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbdc6c4-ea30-45d3-b5c4-9180af125c50 None admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:09.179407 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbdc6c4-ea30-45d3-b5c4-9180af125c50 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:09.179682 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbdc6c4-ea30-45d3-b5c4-9180af125c50 None admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:09.180540 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbdc6c4-ea30-45d3-b5c4-9180af125c50 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:09.182097 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbdc6c4-ea30-45d3-b5c4-9180af125c50 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:09.186833 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda12a38-5735-4c05-9789-a26bb7944b4a None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:09.224914 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f29e7f9-4a28-47a1-9688-b5e50c76ad97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:09.225811 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f29e7f9-4a28-47a1-9688-b5e50c76ad97 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:09.226096 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f29e7f9-4a28-47a1-9688-b5e50c76ad97 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:09.226371 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f29e7f9-4a28-47a1-9688-b5e50c76ad97 None admin] PATH_INFO: `/v3/users/9e991dc7aae347a08469270f3213700f` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:09.239476 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f29e7f9-4a28-47a1-9688-b5e50c76ad97 None admin] RBAC: Authorizing `identity:get_user(user_id=9e991dc7aae347a08469270f3213700f)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:09.241545 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f29e7f9-4a28-47a1-9688-b5e50c76ad97 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:09.247864 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 118/242] 10.222.0.44 () {56 vars in 1177 bytes} [Wed Jun 29 11:06:09 2022] GET /identity/v3/users/9e991dc7aae347a08469270f3213700f => generated 297 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:09.247864 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3da3b35-0920-408e-8753-448a5fdb4f34 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:09.247864 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3da3b35-0920-408e-8753-448a5fdb4f34 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:09.247864 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3da3b35-0920-408e-8753-448a5fdb4f34 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:09.247864 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 119/243] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:06:09 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:06:09.249625 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3da3b35-0920-408e-8753-448a5fdb4f34 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:09.267741 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 125/244] 10.222.0.44 () {60 vars in 1132 bytes} [Wed Jun 29 11:06:08 2022] POST /identity/v3/users => generated 293 bytes in 328 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:06:09.269424 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e98406e-027c-42f2-b9ec-18c391697c87 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:09.269911 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e98406e-027c-42f2-b9ec-18c391697c87 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:09.270383 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e98406e-027c-42f2-b9ec-18c391697c87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:09.282082 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:09.289423 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-1e98406e-027c-42f2-b9ec-18c391697c87 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:06:09.291744 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-1e98406e-027c-42f2-b9ec-18c391697c87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:06:09.394299 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:09.419533 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 126/245] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:06:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 11:06:09.429499 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3da3b35-0920-408e-8753-448a5fdb4f34 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:09.450388 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e98406e-027c-42f2-b9ec-18c391697c87 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:09.519468 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc6e4e0-1697-4f36-bd03-d42bb205cea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:09.520465 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc6e4e0-1697-4f36-bd03-d42bb205cea2 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:09.520926 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc6e4e0-1697-4f36-bd03-d42bb205cea2 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:09.521378 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc6e4e0-1697-4f36-bd03-d42bb205cea2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:09.561291 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e98406e-027c-42f2-b9ec-18c391697c87 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:09.598495 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc6e4e0-1697-4f36-bd03-d42bb205cea2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:09.600922 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc6e4e0-1697-4f36-bd03-d42bb205cea2 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:09.603219 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3be8ab4-dd14-410e-b835-05808717e043 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:09.604047 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3be8ab4-dd14-410e-b835-05808717e043 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:09.604327 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3be8ab4-dd14-410e-b835-05808717e043 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:09.605334 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3be8ab4-dd14-410e-b835-05808717e043 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:09.615252 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3be8ab4-dd14-410e-b835-05808717e043 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 11:06:09.624641 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-d3be8ab4-dd14-410e-b835-05808717e043 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 29 11:06:09.624641 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:06:09.624641 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:06:09.624641 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:06:09.624641 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:06:09.624641 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:06:09.624641 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:06:09.624641 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:06:09.624641 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:06:09.624641 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:06:09.624641 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:06:09.624641 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:06:09.624641 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 11:06:09.624641 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 11:06:09.624641 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 11:06:09.624641 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 11:06:09.625545 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 11:06:09.625545 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 11:06:09.625545 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 11:06:09.625545 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 11:06:09.625545 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:06:09.625545 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:06:09.625545 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:06:09.625545 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:06:09.625545 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:06:09.625545 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:06:09.625545 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:06:09.625545 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jun 29 11:06:09.625545 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:06:09.625545 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:06:09.625545 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:06:09.625545 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:06:09.626336 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:06:09.626336 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:06:09.626336 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:06:09.626336 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:06:09.626336 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 11:06:09.626336 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 11:06:09.626336 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 11:06:09.626336 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 11:06:09.626336 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 11:06:09.626336 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 11:06:09.626336 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 29 11:06:09.626336 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jun 29 11:06:09.626336 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 127/246] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Jun 29 11:06:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 11:06:09.632047 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:09.634280 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3be8ab4-dd14-410e-b835-05808717e043 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:09.686999 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:09.717224 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 120/247] 10.222.0.44 () {58 vars in 1282 bytes} [Wed Jun 29 11:06:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 468 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 29 11:06:09.723137 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc6e4e0-1697-4f36-bd03-d42bb205cea2 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:09.750758 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3be8ab4-dd14-410e-b835-05808717e043 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:09.786122 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d981de-467c-483a-a91a-741baedd9795 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:09.787191 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d981de-467c-483a-a91a-741baedd9795 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:09.787709 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d981de-467c-483a-a91a-741baedd9795 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:09.788253 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d981de-467c-483a-a91a-741baedd9795 None admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:09.789280 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d981de-467c-483a-a91a-741baedd9795 None admin] RBAC: Adding query filter params (name=member) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:06:09.789780 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d981de-467c-483a-a91a-741baedd9795 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:09.791342 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d981de-467c-483a-a91a-741baedd9795 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:09.805772 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 128/248] 10.222.0.44 () {56 vars in 1101 bytes} [Wed Jun 29 11:06:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:09.807329 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d981de-467c-483a-a91a-741baedd9795 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:09.831765 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc6e4e0-1697-4f36-bd03-d42bb205cea2 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:09.858207 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:09.873017 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e62c2a-11b0-4a8d-948f-3237404a53b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:09.874548 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e62c2a-11b0-4a8d-948f-3237404a53b4 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:09.874838 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e62c2a-11b0-4a8d-948f-3237404a53b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:09.875113 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e62c2a-11b0-4a8d-948f-3237404a53b4 None admin] PATH_INFO: `/v3/projects/d36c6c9eb3234e82bb6a8537851eb336` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:09.876557 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e62c2a-11b0-4a8d-948f-3237404a53b4 None admin] RBAC: Authorizing `identity:get_project(project_id=d36c6c9eb3234e82bb6a8537851eb336)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:09.878027 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e62c2a-11b0-4a8d-948f-3237404a53b4 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:09.883858 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 121/249] 10.222.0.44 () {56 vars in 1186 bytes} [Wed Jun 29 11:06:09 2022] GET /identity/v3/projects/d36c6c9eb3234e82bb6a8537851eb336 => generated 302 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:09.887446 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e62c2a-11b0-4a8d-948f-3237404a53b4 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:09.938133 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d981de-467c-483a-a91a-741baedd9795 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:09.950469 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:09.984202 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff3e5be-311b-4320-86ff-886f604b11ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:09.985705 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff3e5be-311b-4320-86ff-886f604b11ac None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:09.986187 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff3e5be-311b-4320-86ff-886f604b11ac None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:09.987224 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff3e5be-311b-4320-86ff-886f604b11ac None admin] PATH_INFO: `/v3/users/bfeaeb065ed045e7b9d3e24a65869673` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:09.989069 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff3e5be-311b-4320-86ff-886f604b11ac None admin] RBAC: Authorizing `identity:get_user(user_id=bfeaeb065ed045e7b9d3e24a65869673)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:09.991897 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff3e5be-311b-4320-86ff-886f604b11ac None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:09.993771 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 129/250] 10.222.0.44 () {56 vars in 1177 bytes} [Wed Jun 29 11:06:09 2022] GET /identity/v3/users/bfeaeb065ed045e7b9d3e24a65869673 => generated 297 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 11:06:10.001491 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff3e5be-311b-4320-86ff-886f604b11ac None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:10.024740 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e62c2a-11b0-4a8d-948f-3237404a53b4 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:10.066086 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de84fcb5-f1ff-4cf9-b5e3-b94ce7f02b2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:10.066816 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de84fcb5-f1ff-4cf9-b5e3-b94ce7f02b2a None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:10.067100 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de84fcb5-f1ff-4cf9-b5e3-b94ce7f02b2a None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:10.067379 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de84fcb5-f1ff-4cf9-b5e3-b94ce7f02b2a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:10.068307 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de84fcb5-f1ff-4cf9-b5e3-b94ce7f02b2a None admin] RBAC: Adding query filter params (role.id=7416219b213f48a4a93d964e67f990fe, scope.project.id=d36c6c9eb3234e82bb6a8537851eb336, user.id=9e991dc7aae347a08469270f3213700f) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:06:10.068624 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de84fcb5-f1ff-4cf9-b5e3-b94ce7f02b2a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:10.070131 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de84fcb5-f1ff-4cf9-b5e3-b94ce7f02b2a None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:10.078320 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 122/251] 10.222.0.44 () {56 vars in 1374 bytes} [Wed Jun 29 11:06:09 2022] GET /identity/v3/role_assignments?user.id=9e991dc7aae347a08469270f3213700f&scope.project.id=d36c6c9eb3234e82bb6a8537851eb336&role.id=7416219b213f48a4a93d964e67f990fe => generated 259 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:10.105460 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87abe8cb-964e-44f1-8018-8f8f2795cfa4 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:10.105792 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87abe8cb-964e-44f1-8018-8f8f2795cfa4 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:10.106030 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87abe8cb-964e-44f1-8018-8f8f2795cfa4 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:10.107671 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 123/252] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:06:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:06:10.114726 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca575cf4-1bd1-4496-ba03-a65211b46528 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:10.115031 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca575cf4-1bd1-4496-ba03-a65211b46528 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:10.115287 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca575cf4-1bd1-4496-ba03-a65211b46528 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:10.133982 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-ca575cf4-1bd1-4496-ba03-a65211b46528 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:06:10.134646 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff3e5be-311b-4320-86ff-886f604b11ac None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:10.135254 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-ca575cf4-1bd1-4496-ba03-a65211b46528 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:06:10.184170 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4165a2c-6137-4713-87d9-b92241ca316e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:10.185443 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4165a2c-6137-4713-87d9-b92241ca316e None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:10.185922 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4165a2c-6137-4713-87d9-b92241ca316e None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:10.186388 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4165a2c-6137-4713-87d9-b92241ca316e None admin] PATH_INFO: `/v3/projects/d36c6c9eb3234e82bb6a8537851eb336` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:10.188702 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4165a2c-6137-4713-87d9-b92241ca316e None admin] RBAC: Authorizing `identity:get_project(project_id=d36c6c9eb3234e82bb6a8537851eb336)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:10.190978 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4165a2c-6137-4713-87d9-b92241ca316e None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:10.197700 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 130/253] 10.222.0.44 () {56 vars in 1186 bytes} [Wed Jun 29 11:06:09 2022] GET /identity/v3/projects/d36c6c9eb3234e82bb6a8537851eb336 => generated 302 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 11:06:10.201973 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4165a2c-6137-4713-87d9-b92241ca316e None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:10.204177 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:10.229547 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 124/254] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:06:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 29 11:06:10.268308 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:10.271671 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca575cf4-1bd1-4496-ba03-a65211b46528 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:10.370174 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4165a2c-6137-4713-87d9-b92241ca316e None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:10.416868 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7775fad6-dd66-4827-ac23-0a18d296c628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:10.417765 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7775fad6-dd66-4827-ac23-0a18d296c628 None admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:10.418050 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7775fad6-dd66-4827-ac23-0a18d296c628 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:10.418332 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7775fad6-dd66-4827-ac23-0a18d296c628 None admin] PATH_INFO: `/v3/projects/d36c6c9eb3234e82bb6a8537851eb336/users/bfeaeb065ed045e7b9d3e24a65869673/roles/e1c88eebe911432cab6a07fbeb84653b` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:10.420375 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7775fad6-dd66-4827-ac23-0a18d296c628 None admin] RBAC: Authorizing `identity:create_grant(project_id=d36c6c9eb3234e82bb6a8537851eb336, user_id=bfeaeb065ed045e7b9d3e24a65869673, role_id=e1c88eebe911432cab6a07fbeb84653b)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:10.421930 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7775fad6-dd66-4827-ac23-0a18d296c628 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:10.457951 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca575cf4-1bd1-4496-ba03-a65211b46528 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:10.491049 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 131/255] 10.222.0.44 () {58 vars in 1439 bytes} [Wed Jun 29 11:06:10 2022] PUT /identity/v3/projects/d36c6c9eb3234e82bb6a8537851eb336/users/bfeaeb065ed045e7b9d3e24a65869673/roles/e1c88eebe911432cab6a07fbeb84653b => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:06:10.509841 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f9b5e8-666d-488e-bb68-7b3afb616738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:10.510682 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f9b5e8-666d-488e-bb68-7b3afb616738 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:10.510968 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f9b5e8-666d-488e-bb68-7b3afb616738 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:10.511245 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f9b5e8-666d-488e-bb68-7b3afb616738 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:10.528061 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f9b5e8-666d-488e-bb68-7b3afb616738 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 11:06:10.536348 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-a6f9b5e8-666d-488e-bb68-7b3afb616738 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 11:06:10.536348 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:06:10.536348 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:06:10.536348 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:06:10.536348 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:06:10.536348 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:06:10.536348 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:06:10.536348 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:06:10.536348 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:06:10.536348 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:06:10.536348 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:06:10.536348 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:06:10.536348 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 11:06:10.536348 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 11:06:10.536348 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 11:06:10.536348 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 11:06:10.537294 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 11:06:10.537294 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 11:06:10.537294 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 11:06:10.537294 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 11:06:10.537294 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:06:10.537294 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:06:10.537294 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:06:10.537294 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:06:10.537294 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:06:10.537294 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:06:10.537294 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:06:10.537294 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jun 29 11:06:10.537294 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:06:10.537294 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:06:10.537294 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:06:10.537294 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:06:10.538146 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:06:10.538146 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:06:10.538146 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:06:10.538146 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:06:10.538146 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 11:06:10.538146 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 11:06:10.538146 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 11:06:10.538146 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 11:06:10.538146 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 11:06:10.538146 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 11:06:10.538146 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 11:06:10.538146 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:06:10.542616 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 125/256] 10.222.0.44 () {56 vars in 1096 bytes} [Wed Jun 29 11:06:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 11:06:10.547347 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7775fad6-dd66-4827-ac23-0a18d296c628 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:10.679791 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7775fad6-dd66-4827-ac23-0a18d296c628 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:10.682317 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:10.718130 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc698aa-3425-4089-86d1-8082f9951fd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:10.718727 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc698aa-3425-4089-86d1-8082f9951fd1 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:10.719012 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc698aa-3425-4089-86d1-8082f9951fd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:10.719288 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc698aa-3425-4089-86d1-8082f9951fd1 None admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:10.720075 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc698aa-3425-4089-86d1-8082f9951fd1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:06:10.720386 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc698aa-3425-4089-86d1-8082f9951fd1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:10.721781 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc698aa-3425-4089-86d1-8082f9951fd1 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:10.732015 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 132/257] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Jun 29 11:06:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:10.737838 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f9b5e8-666d-488e-bb68-7b3afb616738 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:10.771473 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14b1a74-e2b3-4cb4-a6db-e983ce8c88b2 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:10.771799 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14b1a74-e2b3-4cb4-a6db-e983ce8c88b2 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:10.772105 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14b1a74-e2b3-4cb4-a6db-e983ce8c88b2 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:10.773656 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 133/258] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:06:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 29 11:06:10.781164 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c176c120-ba8b-4653-a1bf-aade4ba243fb None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:10.781440 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c176c120-ba8b-4653-a1bf-aade4ba243fb None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:10.781720 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c176c120-ba8b-4653-a1bf-aade4ba243fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:10.804612 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-c176c120-ba8b-4653-a1bf-aade4ba243fb None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:06:10.806134 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-c176c120-ba8b-4653-a1bf-aade4ba243fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:06:10.816223 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:10.882128 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f9b5e8-666d-488e-bb68-7b3afb616738 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:10.908109 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 134/259] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:06:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 29 11:06:10.927938 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717dadde-98e9-43f1-932d-0e44d25cc9f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:10.929613 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717dadde-98e9-43f1-932d-0e44d25cc9f0 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:10.930044 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717dadde-98e9-43f1-932d-0e44d25cc9f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:10.930457 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717dadde-98e9-43f1-932d-0e44d25cc9f0 None admin] PATH_INFO: `/v3/users/03fb807e4be54030b7f888a3962b0e51` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:10.931901 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717dadde-98e9-43f1-932d-0e44d25cc9f0 None admin] RBAC: Authorizing `identity:get_user(user_id=03fb807e4be54030b7f888a3962b0e51)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:10.933568 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717dadde-98e9-43f1-932d-0e44d25cc9f0 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:10.935714 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 126/260] 10.222.0.44 () {56 vars in 1177 bytes} [Wed Jun 29 11:06:10 2022] GET /identity/v3/users/03fb807e4be54030b7f888a3962b0e51 => generated 283 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:10.941367 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c176c120-ba8b-4653-a1bf-aade4ba243fb None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:10.946608 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717dadde-98e9-43f1-932d-0e44d25cc9f0 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:11.114584 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:11.122669 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c176c120-ba8b-4653-a1bf-aade4ba243fb None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:11.182387 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f0d55f-2613-4ba9-add0-0a5bb9c831ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:11.183390 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f0d55f-2613-4ba9-add0-0a5bb9c831ad None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:11.183888 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f0d55f-2613-4ba9-add0-0a5bb9c831ad None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:11.184377 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f0d55f-2613-4ba9-add0-0a5bb9c831ad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:11.189110 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717dadde-98e9-43f1-932d-0e44d25cc9f0 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:11.218881 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:11.262028 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56ddcab-84d9-4db7-b14e-a863b12a21c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:11.262839 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56ddcab-84d9-4db7-b14e-a863b12a21c9 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:11.263120 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56ddcab-84d9-4db7-b14e-a863b12a21c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:11.263389 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56ddcab-84d9-4db7-b14e-a863b12a21c9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:11.285258 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56ddcab-84d9-4db7-b14e-a863b12a21c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 11:06:11.300632 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-d56ddcab-84d9-4db7-b14e-a863b12a21c9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 29 11:06:11.300632 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:06:11.300632 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:06:11.300632 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:06:11.300632 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:06:11.300632 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:06:11.300632 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:06:11.300632 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:06:11.300632 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:06:11.300632 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:06:11.300632 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:06:11.300632 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:06:11.300632 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 11:06:11.300632 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 11:06:11.300632 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 11:06:11.300632 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 11:06:11.301548 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 11:06:11.301548 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 11:06:11.301548 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 11:06:11.301548 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 11:06:11.301548 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:06:11.301548 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:06:11.301548 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:06:11.301548 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:06:11.301548 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:06:11.301548 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:06:11.301548 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:06:11.301548 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jun 29 11:06:11.301548 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:06:11.301548 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:06:11.301548 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:06:11.301548 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:06:11.302338 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:06:11.302338 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:06:11.302338 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:06:11.302338 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:06:11.302338 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 11:06:11.302338 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 11:06:11.302338 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 11:06:11.302338 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 11:06:11.302338 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 11:06:11.302338 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 11:06:11.302338 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 29 11:06:11.302338 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:06:11.302338 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 127/261] 10.222.0.44 () {56 vars in 1114 bytes} [Wed Jun 29 11:06:10 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 356 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:06:11.309205 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56ddcab-84d9-4db7-b14e-a863b12a21c9 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:11.329209 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f0d55f-2613-4ba9-add0-0a5bb9c831ad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:11.330487 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f0d55f-2613-4ba9-add0-0a5bb9c831ad None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:11.536262 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56ddcab-84d9-4db7-b14e-a863b12a21c9 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:11.583529 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d8d774-1ac4-40f9-98db-717e0dd69dbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:11.584239 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d8d774-1ac4-40f9-98db-717e0dd69dbe None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:11.584558 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d8d774-1ac4-40f9-98db-717e0dd69dbe None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:11.585296 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d8d774-1ac4-40f9-98db-717e0dd69dbe None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:11.586059 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d8d774-1ac4-40f9-98db-717e0dd69dbe None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:06:11.586372 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d8d774-1ac4-40f9-98db-717e0dd69dbe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:11.587753 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d8d774-1ac4-40f9-98db-717e0dd69dbe None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:11.597923 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 128/262] 10.222.0.44 () {56 vars in 1111 bytes} [Wed Jun 29 11:06:11 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:11.607367 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d8d774-1ac4-40f9-98db-717e0dd69dbe None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:11.617613 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:11.642632 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 135/263] 10.222.0.44 () {58 vars in 1282 bytes} [Wed Jun 29 11:06:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 702 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 29 11:06:11.649006 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f0d55f-2613-4ba9-add0-0a5bb9c831ad None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:11.672169 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:11.802689 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d8d774-1ac4-40f9-98db-717e0dd69dbe None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:11.864308 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d402cab-b46b-4101-8477-57663c7e4979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:11.865185 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d402cab-b46b-4101-8477-57663c7e4979 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:11.865467 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d402cab-b46b-4101-8477-57663c7e4979 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:11.865781 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d402cab-b46b-4101-8477-57663c7e4979 None admin] PATH_INFO: `/v3/users/03fb807e4be54030b7f888a3962b0e51` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:11.869262 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d402cab-b46b-4101-8477-57663c7e4979 None admin] RBAC: Authorizing `identity:get_user(user_id=03fb807e4be54030b7f888a3962b0e51)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:11.870697 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d402cab-b46b-4101-8477-57663c7e4979 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:11.872552 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 129/264] 10.222.0.44 () {56 vars in 1177 bytes} [Wed Jun 29 11:06:11 2022] GET /identity/v3/users/03fb807e4be54030b7f888a3962b0e51 => generated 283 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:11.878068 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d402cab-b46b-4101-8477-57663c7e4979 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:11.900158 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f0d55f-2613-4ba9-add0-0a5bb9c831ad None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:11.960772 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c7b442-32fe-4483-8fa0-2d116a3fc45c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:11.961648 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c7b442-32fe-4483-8fa0-2d116a3fc45c None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:11.961935 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c7b442-32fe-4483-8fa0-2d116a3fc45c None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:11.962209 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c7b442-32fe-4483-8fa0-2d116a3fc45c None admin] PATH_INFO: `/v3/projects/d36c6c9eb3234e82bb6a8537851eb336` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:11.963463 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c7b442-32fe-4483-8fa0-2d116a3fc45c None admin] RBAC: Authorizing `identity:get_project(project_id=d36c6c9eb3234e82bb6a8537851eb336)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:11.964948 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c7b442-32fe-4483-8fa0-2d116a3fc45c None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:11.966976 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 136/265] 10.222.0.44 () {56 vars in 1186 bytes} [Wed Jun 29 11:06:11 2022] GET /identity/v3/projects/d36c6c9eb3234e82bb6a8537851eb336 => generated 302 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:11.972655 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c7b442-32fe-4483-8fa0-2d116a3fc45c None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:12.011984 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d402cab-b46b-4101-8477-57663c7e4979 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:12.020174 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:12.049162 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f356f95b-69b7-43a5-bb50-92729463b655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:12.049815 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f356f95b-69b7-43a5-bb50-92729463b655 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:12.050097 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f356f95b-69b7-43a5-bb50-92729463b655 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:12.060048 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f356f95b-69b7-43a5-bb50-92729463b655 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:12.079589 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:12.136276 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f356f95b-69b7-43a5-bb50-92729463b655 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:12.137909 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f356f95b-69b7-43a5-bb50-92729463b655 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:12.216267 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c7b442-32fe-4483-8fa0-2d116a3fc45c None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:12.269749 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 130/266] 10.222.0.44 () {58 vars in 1282 bytes} [Wed Jun 29 11:06:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 29 11:06:12.275245 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f356f95b-69b7-43a5-bb50-92729463b655 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:12.292562 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1444319c-4bac-47ef-881c-6aa107e9a4f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:12.293252 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1444319c-4bac-47ef-881c-6aa107e9a4f4 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:12.293545 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1444319c-4bac-47ef-881c-6aa107e9a4f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:12.293831 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1444319c-4bac-47ef-881c-6aa107e9a4f4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:12.294176 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:12.294690 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1444319c-4bac-47ef-881c-6aa107e9a4f4 None admin] RBAC: Adding query filter params (role.id=e95c2ce63e114cc19a9dda6ad21e794f, scope.project.id=d36c6c9eb3234e82bb6a8537851eb336, user.id=03fb807e4be54030b7f888a3962b0e51) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:06:12.295012 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1444319c-4bac-47ef-881c-6aa107e9a4f4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:12.296546 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1444319c-4bac-47ef-881c-6aa107e9a4f4 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:12.308066 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 137/267] 10.222.0.44 () {56 vars in 1374 bytes} [Wed Jun 29 11:06:11 2022] GET /identity/v3/role_assignments?user.id=03fb807e4be54030b7f888a3962b0e51&scope.project.id=d36c6c9eb3234e82bb6a8537851eb336&role.id=e95c2ce63e114cc19a9dda6ad21e794f => generated 608 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:12.450270 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f356f95b-69b7-43a5-bb50-92729463b655 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:12.501514 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac005b9-2443-455f-8e44-6472af1aa80c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:12.502407 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac005b9-2443-455f-8e44-6472af1aa80c None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:12.502687 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac005b9-2443-455f-8e44-6472af1aa80c None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:12.502958 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac005b9-2443-455f-8e44-6472af1aa80c None admin] PATH_INFO: `/v3/projects/d36c6c9eb3234e82bb6a8537851eb336` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:12.504226 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac005b9-2443-455f-8e44-6472af1aa80c None admin] RBAC: Authorizing `identity:get_project(project_id=d36c6c9eb3234e82bb6a8537851eb336)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:12.505689 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac005b9-2443-455f-8e44-6472af1aa80c None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:12.509525 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 131/268] 10.222.0.44 () {56 vars in 1186 bytes} [Wed Jun 29 11:06:12 2022] GET /identity/v3/projects/d36c6c9eb3234e82bb6a8537851eb336 => generated 302 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:12.514488 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1444319c-4bac-47ef-881c-6aa107e9a4f4 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:12.670706 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1444319c-4bac-47ef-881c-6aa107e9a4f4 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:12.698752 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:12.706483 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c2dbc3-0829-408a-a139-8a03df151f43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:12.707083 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c2dbc3-0829-408a-a139-8a03df151f43 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:12.707366 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c2dbc3-0829-408a-a139-8a03df151f43 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:12.707660 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c2dbc3-0829-408a-a139-8a03df151f43 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:12.708529 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c2dbc3-0829-408a-a139-8a03df151f43 None admin] RBAC: Adding query filter params (role.id=fc91ce4ad4a44cc3b05d0936754e87ec, scope.project.id=d36c6c9eb3234e82bb6a8537851eb336, user.id=03fb807e4be54030b7f888a3962b0e51) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:06:12.708841 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c2dbc3-0829-408a-a139-8a03df151f43 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:12.710873 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c2dbc3-0829-408a-a139-8a03df151f43 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:12.720398 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 138/269] 10.222.0.44 () {56 vars in 1374 bytes} [Wed Jun 29 11:06:12 2022] GET /identity/v3/role_assignments?user.id=03fb807e4be54030b7f888a3962b0e51&scope.project.id=d36c6c9eb3234e82bb6a8537851eb336&role.id=fc91ce4ad4a44cc3b05d0936754e87ec => generated 608 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:12.762158 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e6cb14-6e06-41a4-b239-e22961645c3a None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:12.762663 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e6cb14-6e06-41a4-b239-e22961645c3a None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:12.762736 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e6cb14-6e06-41a4-b239-e22961645c3a None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:12.764432 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 132/270] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:06:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:06:12.769669 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ed4a42-5a38-4a02-9266-f7a9023f6c93 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:12.769953 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ed4a42-5a38-4a02-9266-f7a9023f6c93 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:12.770231 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ed4a42-5a38-4a02-9266-f7a9023f6c93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:12.796072 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-89ed4a42-5a38-4a02-9266-f7a9023f6c93 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:06:12.797512 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-89ed4a42-5a38-4a02-9266-f7a9023f6c93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:06:12.820965 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722b3bad-91a9-405c-9648-8111d79e1243 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:12.821263 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722b3bad-91a9-405c-9648-8111d79e1243 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:12.821515 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722b3bad-91a9-405c-9648-8111d79e1243 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:12.824926 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 133/271] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:06:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:06:12.835789 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ba06c3-07e8-4de6-9993-58beb8734999 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:12.836116 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ba06c3-07e8-4de6-9993-58beb8734999 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:12.836382 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ba06c3-07e8-4de6-9993-58beb8734999 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:12.864654 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-f5ba06c3-07e8-4de6-9993-58beb8734999 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:06:12.865912 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-f5ba06c3-07e8-4de6-9993-58beb8734999 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:06:12.890773 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 139/272] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:06:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 11:06:12.931054 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ed4a42-5a38-4a02-9266-f7a9023f6c93 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:12.993532 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 134/273] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:06:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 11:06:13.024440 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ba06c3-07e8-4de6-9993-58beb8734999 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:13.026730 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:13.065361 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ed4a42-5a38-4a02-9266-f7a9023f6c93 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:13.089649 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:13.102280 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb32134e-932f-4122-96c3-a523274b0f13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:13.103427 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb32134e-932f-4122-96c3-a523274b0f13 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:13.103895 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb32134e-932f-4122-96c3-a523274b0f13 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:13.104331 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb32134e-932f-4122-96c3-a523274b0f13 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:13.106274 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb32134e-932f-4122-96c3-a523274b0f13 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:13.108028 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb32134e-932f-4122-96c3-a523274b0f13 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:13.110340 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 140/274] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jun 29 11:06:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:13.115254 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb32134e-932f-4122-96c3-a523274b0f13 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:13.142428 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ba06c3-07e8-4de6-9993-58beb8734999 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:13.181364 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d460285b-c24e-4ca0-8fc1-87745989bd7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:13.182246 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d460285b-c24e-4ca0-8fc1-87745989bd7e None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:13.182526 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d460285b-c24e-4ca0-8fc1-87745989bd7e None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:13.182846 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d460285b-c24e-4ca0-8fc1-87745989bd7e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:13.194456 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d460285b-c24e-4ca0-8fc1-87745989bd7e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 11:06:13.202964 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-d460285b-c24e-4ca0-8fc1-87745989bd7e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 29 11:06:13.202964 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:06:13.202964 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:06:13.202964 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:06:13.202964 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:06:13.202964 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:06:13.202964 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:06:13.202964 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:06:13.202964 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:06:13.202964 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:06:13.202964 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:06:13.202964 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:06:13.202964 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 11:06:13.202964 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 11:06:13.202964 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 11:06:13.202964 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 11:06:13.203939 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 11:06:13.203939 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 11:06:13.203939 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 11:06:13.203939 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 11:06:13.203939 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:06:13.203939 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:06:13.203939 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:06:13.203939 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:06:13.203939 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:06:13.203939 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:06:13.203939 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:06:13.203939 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jun 29 11:06:13.203939 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:06:13.203939 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:06:13.203939 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:06:13.203939 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:06:13.204739 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:06:13.204739 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:06:13.204739 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:06:13.204739 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:06:13.204739 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 11:06:13.204739 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 11:06:13.204739 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 11:06:13.204739 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 11:06:13.204739 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 11:06:13.204739 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 11:06:13.204739 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 29 11:06:13.204739 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:06:13.204739 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 135/275] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Jun 29 11:06:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 11:06:13.210710 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d460285b-c24e-4ca0-8fc1-87745989bd7e None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:13.224597 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb32134e-932f-4122-96c3-a523274b0f13 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:13.260194 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ee855a-bb3e-48cf-8e48-c78640a77434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:13.261188 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ee855a-bb3e-48cf-8e48-c78640a77434 None admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:13.261690 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ee855a-bb3e-48cf-8e48-c78640a77434 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:13.262178 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ee855a-bb3e-48cf-8e48-c78640a77434 None admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:13.263082 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ee855a-bb3e-48cf-8e48-c78640a77434 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:13.264759 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ee855a-bb3e-48cf-8e48-c78640a77434 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:13.282408 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:13.323447 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d460285b-c24e-4ca0-8fc1-87745989bd7e None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:13.353335 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 141/276] 10.222.0.44 () {60 vars in 1132 bytes} [Wed Jun 29 11:06:13 2022] POST /identity/v3/users => generated 293 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:06:13.354514 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd3f404-8487-46e4-b231-a3d23e4ba512 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:13.354800 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd3f404-8487-46e4-b231-a3d23e4ba512 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:13.355072 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd3f404-8487-46e4-b231-a3d23e4ba512 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:13.359976 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 142/277] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:06:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:06:13.361208 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb45b77-2ef4-4f51-9c97-1b2d9d8f578a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:13.361943 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb45b77-2ef4-4f51-9c97-1b2d9d8f578a None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:13.362238 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb45b77-2ef4-4f51-9c97-1b2d9d8f578a None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:13.362518 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb45b77-2ef4-4f51-9c97-1b2d9d8f578a None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:13.363297 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb45b77-2ef4-4f51-9c97-1b2d9d8f578a None admin] RBAC: Adding query filter params (name=member) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:06:13.363655 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb45b77-2ef4-4f51-9c97-1b2d9d8f578a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:13.365233 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb45b77-2ef4-4f51-9c97-1b2d9d8f578a None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:13.369457 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04839492-efb0-46d0-a7be-6fa6055fb61f None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:13.369745 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04839492-efb0-46d0-a7be-6fa6055fb61f None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:13.370029 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04839492-efb0-46d0-a7be-6fa6055fb61f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:13.384317 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 136/278] 10.222.0.44 () {56 vars in 1101 bytes} [Wed Jun 29 11:06:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 11:06:13.391995 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb45b77-2ef4-4f51-9c97-1b2d9d8f578a None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:13.395153 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-04839492-efb0-46d0-a7be-6fa6055fb61f None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:06:13.396406 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-04839492-efb0-46d0-a7be-6fa6055fb61f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:06:13.448904 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:13.476498 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 143/279] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:06:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 11:06:13.498439 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04839492-efb0-46d0-a7be-6fa6055fb61f None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:13.540048 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb45b77-2ef4-4f51-9c97-1b2d9d8f578a None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:13.567634 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:13.581862 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b20852-8735-4581-98f5-1b845f1cec86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:13.583664 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b20852-8735-4581-98f5-1b845f1cec86 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:13.584042 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b20852-8735-4581-98f5-1b845f1cec86 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:13.584323 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b20852-8735-4581-98f5-1b845f1cec86 None admin] PATH_INFO: `/v3/users/1d6218cc192148cfb5604f670e863c20` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:13.603304 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b20852-8735-4581-98f5-1b845f1cec86 None admin] RBAC: Authorizing `identity:get_user(user_id=1d6218cc192148cfb5604f670e863c20)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:13.604946 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b20852-8735-4581-98f5-1b845f1cec86 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:13.606566 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 137/280] 10.222.0.44 () {56 vars in 1177 bytes} [Wed Jun 29 11:06:13 2022] GET /identity/v3/users/1d6218cc192148cfb5604f670e863c20 => generated 293 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 11:06:13.607822 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae0f892-ab7b-456e-bdf6-3e10d8cb060b None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:13.608164 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae0f892-ab7b-456e-bdf6-3e10d8cb060b None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:13.608442 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae0f892-ab7b-456e-bdf6-3e10d8cb060b None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:13.609831 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 138/281] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:06:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:06:13.613874 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae0f892-ab7b-456e-bdf6-3e10d8cb060b None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:13.636135 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04839492-efb0-46d0-a7be-6fa6055fb61f None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:13.684305 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7f6c80-1784-4379-9be2-f3ab6d369915 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:13.685780 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7f6c80-1784-4379-9be2-f3ab6d369915 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:13.686063 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7f6c80-1784-4379-9be2-f3ab6d369915 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:13.686337 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7f6c80-1784-4379-9be2-f3ab6d369915 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:13.700143 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7f6c80-1784-4379-9be2-f3ab6d369915 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 11:06:13.708318 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-6e7f6c80-1784-4379-9be2-f3ab6d369915 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 29 11:06:13.708318 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:06:13.708318 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:06:13.708318 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:06:13.708318 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:06:13.708318 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:06:13.708318 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:06:13.708318 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:06:13.708318 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:06:13.708318 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:06:13.708318 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:06:13.708318 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:06:13.708318 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 11:06:13.708318 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 11:06:13.708318 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 11:06:13.708318 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 11:06:13.709258 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 11:06:13.709258 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 11:06:13.709258 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 11:06:13.709258 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 11:06:13.709258 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:06:13.709258 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:06:13.709258 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:06:13.709258 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:06:13.709258 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:06:13.709258 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:06:13.709258 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:06:13.709258 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jun 29 11:06:13.709258 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:06:13.709258 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:06:13.709258 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:06:13.709258 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:06:13.710853 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:06:13.710853 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:06:13.710853 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:06:13.710853 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:06:13.710853 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 11:06:13.710853 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 11:06:13.710853 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 11:06:13.710853 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 11:06:13.710853 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 11:06:13.710853 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 11:06:13.710853 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 29 11:06:13.710853 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jun 29 11:06:13.713214 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 144/282] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Jun 29 11:06:13 2022] GET /identity/v3/roles/reader => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 11:06:13.714565 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f47cb0-ed4e-4aeb-87f7-195456cf7457 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:13.714853 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f47cb0-ed4e-4aeb-87f7-195456cf7457 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:13.715129 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f47cb0-ed4e-4aeb-87f7-195456cf7457 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:13.734543 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-89f47cb0-ed4e-4aeb-87f7-195456cf7457 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:06:13.735995 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-89f47cb0-ed4e-4aeb-87f7-195456cf7457 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:06:13.748283 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae0f892-ab7b-456e-bdf6-3e10d8cb060b None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:13.772086 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:13.788621 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9375db-5773-4416-aaf1-eaec1dc36ec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:13.791555 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9375db-5773-4416-aaf1-eaec1dc36ec4 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:13.791555 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9375db-5773-4416-aaf1-eaec1dc36ec4 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:13.791555 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9375db-5773-4416-aaf1-eaec1dc36ec4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:13.792153 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9375db-5773-4416-aaf1-eaec1dc36ec4 None admin] RBAC: Adding query filter params (role.id=e1c88eebe911432cab6a07fbeb84653b, user.id=1d6218cc192148cfb5604f670e863c20, scope.system=all) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:06:13.792762 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9375db-5773-4416-aaf1-eaec1dc36ec4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:13.794382 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9375db-5773-4416-aaf1-eaec1dc36ec4 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:13.801394 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 139/283] 10.222.0.44 () {56 vars in 1308 bytes} [Wed Jun 29 11:06:13 2022] GET /identity/v3/role_assignments?user.id=1d6218cc192148cfb5604f670e863c20&scope.system=all&role.id=e1c88eebe911432cab6a07fbeb84653b => generated 226 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:13.802754 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9375db-5773-4416-aaf1-eaec1dc36ec4 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:13.821450 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 145/284] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:06:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 11:06:13.846185 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f47cb0-ed4e-4aeb-87f7-195456cf7457 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:13.873851 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:13.983674 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9375db-5773-4416-aaf1-eaec1dc36ec4 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:14.024147 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f47cb0-ed4e-4aeb-87f7-195456cf7457 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:14.029810 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4538bc99-f8e2-4706-8229-eef22b3de345 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:14.030675 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4538bc99-f8e2-4706-8229-eef22b3de345 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:14.031099 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4538bc99-f8e2-4706-8229-eef22b3de345 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:14.031511 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4538bc99-f8e2-4706-8229-eef22b3de345 None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:14.032427 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4538bc99-f8e2-4706-8229-eef22b3de345 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:06:14.032907 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4538bc99-f8e2-4706-8229-eef22b3de345 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:14.034299 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4538bc99-f8e2-4706-8229-eef22b3de345 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:14.048132 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 140/285] 10.222.0.44 () {56 vars in 1101 bytes} [Wed Jun 29 11:06:13 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:14.052880 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4538bc99-f8e2-4706-8229-eef22b3de345 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:14.069359 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be21e772-adce-4032-88fa-9b15d731f78c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:14.070415 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be21e772-adce-4032-88fa-9b15d731f78c None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:14.071302 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be21e772-adce-4032-88fa-9b15d731f78c None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:14.071764 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be21e772-adce-4032-88fa-9b15d731f78c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:14.084195 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be21e772-adce-4032-88fa-9b15d731f78c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 11:06:14.093289 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-be21e772-adce-4032-88fa-9b15d731f78c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 29 11:06:14.093289 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:06:14.093289 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:06:14.093289 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:06:14.093289 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:06:14.093289 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:06:14.093289 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:06:14.093289 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:06:14.093289 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:06:14.093289 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:06:14.093289 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:06:14.093289 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:06:14.093289 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 11:06:14.093289 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 11:06:14.093289 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 11:06:14.093289 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 11:06:14.094430 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 11:06:14.094430 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 11:06:14.094430 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 11:06:14.094430 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 11:06:14.094430 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:06:14.094430 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:06:14.094430 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:06:14.094430 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:06:14.094430 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:06:14.094430 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:06:14.094430 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:06:14.094430 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jun 29 11:06:14.094430 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:06:14.094430 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:06:14.094430 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:06:14.094430 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:06:14.095423 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:06:14.095423 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:06:14.095423 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:06:14.095423 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:06:14.095423 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 11:06:14.095423 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 11:06:14.095423 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 11:06:14.095423 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 11:06:14.095423 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 11:06:14.095423 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 11:06:14.095423 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 29 11:06:14.095423 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jun 29 11:06:14.097854 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 146/286] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Jun 29 11:06:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 11:06:14.106816 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be21e772-adce-4032-88fa-9b15d731f78c None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:14.140805 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:14.224050 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be21e772-adce-4032-88fa-9b15d731f78c None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:14.269152 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170b498b-8f0f-4d9e-95fb-a3e333ea01bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:14.269982 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170b498b-8f0f-4d9e-95fb-a3e333ea01bf None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:14.270363 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170b498b-8f0f-4d9e-95fb-a3e333ea01bf None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:14.270738 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170b498b-8f0f-4d9e-95fb-a3e333ea01bf None admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:14.271634 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170b498b-8f0f-4d9e-95fb-a3e333ea01bf None admin] RBAC: Adding query filter params (name=member) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:06:14.272596 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170b498b-8f0f-4d9e-95fb-a3e333ea01bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:14.274141 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170b498b-8f0f-4d9e-95fb-a3e333ea01bf None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:14.282697 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4538bc99-f8e2-4706-8229-eef22b3de345 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:14.290726 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 147/287] 10.222.0.44 () {56 vars in 1101 bytes} [Wed Jun 29 11:06:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:14.295964 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170b498b-8f0f-4d9e-95fb-a3e333ea01bf None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:14.298308 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:14.350211 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9415789e-1934-4b81-9bd3-12505355b801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:14.351089 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9415789e-1934-4b81-9bd3-12505355b801 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:14.351371 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9415789e-1934-4b81-9bd3-12505355b801 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:14.351648 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9415789e-1934-4b81-9bd3-12505355b801 None admin] PATH_INFO: `/v3/users/9e991dc7aae347a08469270f3213700f` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:14.360673 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9415789e-1934-4b81-9bd3-12505355b801 None admin] RBAC: Authorizing `identity:get_user(user_id=9e991dc7aae347a08469270f3213700f)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:14.362148 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9415789e-1934-4b81-9bd3-12505355b801 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:14.364153 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 141/288] 10.222.0.44 () {56 vars in 1177 bytes} [Wed Jun 29 11:06:14 2022] GET /identity/v3/users/9e991dc7aae347a08469270f3213700f => generated 297 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:14.369798 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9415789e-1934-4b81-9bd3-12505355b801 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:14.415713 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:14.429035 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170b498b-8f0f-4d9e-95fb-a3e333ea01bf None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:14.467493 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6f7ae2-4899-4c23-81d8-9ff59a222d58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:14.468753 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6f7ae2-4899-4c23-81d8-9ff59a222d58 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:14.469390 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6f7ae2-4899-4c23-81d8-9ff59a222d58 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:14.471174 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6f7ae2-4899-4c23-81d8-9ff59a222d58 None admin] PATH_INFO: `/v3/users/bfeaeb065ed045e7b9d3e24a65869673` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:14.472582 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6f7ae2-4899-4c23-81d8-9ff59a222d58 None admin] RBAC: Authorizing `identity:get_user(user_id=bfeaeb065ed045e7b9d3e24a65869673)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:14.476442 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6f7ae2-4899-4c23-81d8-9ff59a222d58 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:14.478424 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 148/289] 10.222.0.44 () {56 vars in 1177 bytes} [Wed Jun 29 11:06:14 2022] GET /identity/v3/users/bfeaeb065ed045e7b9d3e24a65869673 => generated 297 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:14.484859 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6f7ae2-4899-4c23-81d8-9ff59a222d58 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:14.552228 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9415789e-1934-4b81-9bd3-12505355b801 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:14.594228 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb80dded-1e55-49f6-a36c-6a573d57065c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:14.595088 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb80dded-1e55-49f6-a36c-6a573d57065c None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:14.595367 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb80dded-1e55-49f6-a36c-6a573d57065c None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:14.595639 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb80dded-1e55-49f6-a36c-6a573d57065c None admin] PATH_INFO: `/v3/projects/d36c6c9eb3234e82bb6a8537851eb336` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:14.597862 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb80dded-1e55-49f6-a36c-6a573d57065c None admin] RBAC: Authorizing `identity:get_project(project_id=d36c6c9eb3234e82bb6a8537851eb336)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:14.599311 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb80dded-1e55-49f6-a36c-6a573d57065c None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:14.606269 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 142/290] 10.222.0.44 () {56 vars in 1186 bytes} [Wed Jun 29 11:06:14 2022] GET /identity/v3/projects/d36c6c9eb3234e82bb6a8537851eb336 => generated 302 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 11:06:14.608233 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb80dded-1e55-49f6-a36c-6a573d57065c None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:14.624050 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6f7ae2-4899-4c23-81d8-9ff59a222d58 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:14.687097 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:14.687277 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e20a58-692b-4540-9c86-a5c925ed72b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:14.687932 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e20a58-692b-4540-9c86-a5c925ed72b1 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:14.688134 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e20a58-692b-4540-9c86-a5c925ed72b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:14.688388 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e20a58-692b-4540-9c86-a5c925ed72b1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:14.740247 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:14.799545 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e20a58-692b-4540-9c86-a5c925ed72b1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:14.801064 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e20a58-692b-4540-9c86-a5c925ed72b1 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:14.806620 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb80dded-1e55-49f6-a36c-6a573d57065c None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:14.892600 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c77a67-d66a-4ab5-ab15-ea8f177d7f5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:14.893516 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c77a67-d66a-4ab5-ab15-ea8f177d7f5a None admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:14.893800 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c77a67-d66a-4ab5-ab15-ea8f177d7f5a None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:14.894069 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c77a67-d66a-4ab5-ab15-ea8f177d7f5a None admin] PATH_INFO: `/v3/projects/d36c6c9eb3234e82bb6a8537851eb336/users/9e991dc7aae347a08469270f3213700f/roles/7416219b213f48a4a93d964e67f990fe` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:14.900594 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c77a67-d66a-4ab5-ab15-ea8f177d7f5a None admin] RBAC: Authorizing `identity:create_grant(project_id=d36c6c9eb3234e82bb6a8537851eb336, user_id=9e991dc7aae347a08469270f3213700f, role_id=7416219b213f48a4a93d964e67f990fe)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:14.902050 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c77a67-d66a-4ab5-ab15-ea8f177d7f5a None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:14.902967 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 149/291] 10.222.0.44 () {58 vars in 1282 bytes} [Wed Jun 29 11:06:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 29 11:06:14.912893 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e20a58-692b-4540-9c86-a5c925ed72b1 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:14.986307 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 143/292] 10.222.0.44 () {58 vars in 1439 bytes} [Wed Jun 29 11:06:14 2022] PUT /identity/v3/projects/d36c6c9eb3234e82bb6a8537851eb336/users/9e991dc7aae347a08469270f3213700f/roles/7416219b213f48a4a93d964e67f990fe => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:06:14.992227 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:15.015294 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e20a58-692b-4540-9c86-a5c925ed72b1 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:15.052349 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b614a3-ebae-43ba-8f89-545d95f6a3f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:15.053255 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b614a3-ebae-43ba-8f89-545d95f6a3f1 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:15.053534 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b614a3-ebae-43ba-8f89-545d95f6a3f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:15.053808 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b614a3-ebae-43ba-8f89-545d95f6a3f1 None admin] PATH_INFO: `/v3/projects/d36c6c9eb3234e82bb6a8537851eb336` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:15.055084 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b614a3-ebae-43ba-8f89-545d95f6a3f1 None admin] RBAC: Authorizing `identity:get_project(project_id=d36c6c9eb3234e82bb6a8537851eb336)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:15.056689 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b614a3-ebae-43ba-8f89-545d95f6a3f1 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:15.058702 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 150/293] 10.222.0.44 () {56 vars in 1186 bytes} [Wed Jun 29 11:06:14 2022] GET /identity/v3/projects/d36c6c9eb3234e82bb6a8537851eb336 => generated 302 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:15.064919 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c77a67-d66a-4ab5-ab15-ea8f177d7f5a None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:15.237258 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c77a67-d66a-4ab5-ab15-ea8f177d7f5a None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:15.241547 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:15.277714 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ec1aa8-efb1-4a3b-8c30-b650a8074f21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:15.278369 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ec1aa8-efb1-4a3b-8c30-b650a8074f21 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:15.278651 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ec1aa8-efb1-4a3b-8c30-b650a8074f21 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:15.278930 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ec1aa8-efb1-4a3b-8c30-b650a8074f21 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:15.279755 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ec1aa8-efb1-4a3b-8c30-b650a8074f21 None admin] RBAC: Adding query filter params (role.id=e1c88eebe911432cab6a07fbeb84653b, scope.project.id=d36c6c9eb3234e82bb6a8537851eb336, user.id=bfeaeb065ed045e7b9d3e24a65869673) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:06:15.280112 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ec1aa8-efb1-4a3b-8c30-b650a8074f21 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:15.281531 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ec1aa8-efb1-4a3b-8c30-b650a8074f21 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:15.288033 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 144/294] 10.222.0.44 () {56 vars in 1374 bytes} [Wed Jun 29 11:06:15 2022] GET /identity/v3/role_assignments?user.id=bfeaeb065ed045e7b9d3e24a65869673&scope.project.id=d36c6c9eb3234e82bb6a8537851eb336&role.id=e1c88eebe911432cab6a07fbeb84653b => generated 608 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:15.788375 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470f1fc8-f810-4230-bb54-5b7373d9cad8 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:15.788882 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470f1fc8-f810-4230-bb54-5b7373d9cad8 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:15.788956 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470f1fc8-f810-4230-bb54-5b7373d9cad8 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:15.790559 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 151/295] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:06:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:06:15.795669 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2d505e-681e-48ce-ac37-d843c844ebcf None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:15.795989 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2d505e-681e-48ce-ac37-d843c844ebcf None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:15.796268 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2d505e-681e-48ce-ac37-d843c844ebcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:15.814109 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-fa2d505e-681e-48ce-ac37-d843c844ebcf None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:06:15.815288 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-fa2d505e-681e-48ce-ac37-d843c844ebcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:06:15.872575 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1576a123-ed51-4371-b261-0e6f592214dc None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:15.872877 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1576a123-ed51-4371-b261-0e6f592214dc None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:15.873113 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1576a123-ed51-4371-b261-0e6f592214dc None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:15.874858 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 152/296] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:06:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 29 11:06:15.881968 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf7a2b1-6e2b-42ea-b231-06889666cea6 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:15.882349 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf7a2b1-6e2b-42ea-b231-06889666cea6 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:15.882722 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf7a2b1-6e2b-42ea-b231-06889666cea6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:15.895297 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 145/297] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:06:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 29 11:06:15.903687 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-0cf7a2b1-6e2b-42ea-b231-06889666cea6 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:06:15.905098 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-0cf7a2b1-6e2b-42ea-b231-06889666cea6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:06:15.926483 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2d505e-681e-48ce-ac37-d843c844ebcf None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:15.992027 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 153/298] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:06:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 11:06:16.015708 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf7a2b1-6e2b-42ea-b231-06889666cea6 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:16.033344 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:16.037440 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2d505e-681e-48ce-ac37-d843c844ebcf None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:16.075500 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fb8645-80e4-440e-b095-b037fd7edbf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:16.075500 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fb8645-80e4-440e-b095-b037fd7edbf4 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:16.075500 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fb8645-80e4-440e-b095-b037fd7edbf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:16.075500 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fb8645-80e4-440e-b095-b037fd7edbf4 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:16.087861 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:16.090492 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fb8645-80e4-440e-b095-b037fd7edbf4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 11:06:16.098452 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-11fb8645-80e4-440e-b095-b037fd7edbf4 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 29 11:06:16.098452 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:06:16.098452 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:06:16.098452 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:06:16.098452 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:06:16.098452 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:06:16.098452 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:06:16.098452 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:06:16.098452 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:06:16.098452 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:06:16.098452 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:06:16.098452 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:06:16.098452 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 11:06:16.098452 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 11:06:16.098452 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 11:06:16.098452 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 11:06:16.099577 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 11:06:16.099577 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 11:06:16.099577 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 11:06:16.099577 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 11:06:16.099577 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:06:16.099577 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:06:16.099577 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:06:16.099577 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:06:16.099577 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:06:16.099577 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:06:16.099577 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:06:16.099577 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jun 29 11:06:16.099577 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:06:16.099577 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:06:16.099577 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:06:16.099577 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:06:16.100557 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:06:16.100557 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:06:16.100557 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:06:16.100557 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:06:16.100557 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 11:06:16.100557 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 11:06:16.100557 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 11:06:16.100557 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 11:06:16.100557 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 11:06:16.100557 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 11:06:16.100557 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 29 11:06:16.100557 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:06:16.100557 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 146/299] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Jun 29 11:06:15 2022] GET /identity/v3/roles/reader => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:06:16.105696 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fb8645-80e4-440e-b095-b037fd7edbf4 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:16.137933 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf7a2b1-6e2b-42ea-b231-06889666cea6 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:16.174847 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894e8d33-9bae-4722-ab1c-466062ec6b81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:16.175906 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894e8d33-9bae-4722-ab1c-466062ec6b81 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:16.176341 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894e8d33-9bae-4722-ab1c-466062ec6b81 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:16.176786 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894e8d33-9bae-4722-ab1c-466062ec6b81 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:16.178945 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894e8d33-9bae-4722-ab1c-466062ec6b81 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:16.180656 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894e8d33-9bae-4722-ab1c-466062ec6b81 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:16.187041 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 154/300] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jun 29 11:06:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:16.188153 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894e8d33-9bae-4722-ab1c-466062ec6b81 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:16.210689 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fb8645-80e4-440e-b095-b037fd7edbf4 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:16.245700 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8c20b5-85c7-4c49-ac93-eb929d527cb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:16.246362 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8c20b5-85c7-4c49-ac93-eb929d527cb8 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:16.246650 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8c20b5-85c7-4c49-ac93-eb929d527cb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:16.246929 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8c20b5-85c7-4c49-ac93-eb929d527cb8 None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:16.247677 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8c20b5-85c7-4c49-ac93-eb929d527cb8 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:06:16.248034 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8c20b5-85c7-4c49-ac93-eb929d527cb8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:16.249407 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8c20b5-85c7-4c49-ac93-eb929d527cb8 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:16.258552 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 147/301] 10.222.0.44 () {56 vars in 1101 bytes} [Wed Jun 29 11:06:16 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:16.264033 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4229227e-342f-45fe-b982-70a2354968b0 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:16.264328 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4229227e-342f-45fe-b982-70a2354968b0 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:16.264611 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4229227e-342f-45fe-b982-70a2354968b0 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:16.265872 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 148/302] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:06:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:06:16.267089 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4229227e-342f-45fe-b982-70a2354968b0 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:16.290785 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:16.300345 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894e8d33-9bae-4722-ab1c-466062ec6b81 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:16.326349 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:16.337240 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f24737-c626-4571-bbd6-1ce3078d7ea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:16.338472 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f24737-c626-4571-bbd6-1ce3078d7ea6 None admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:16.338472 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f24737-c626-4571-bbd6-1ce3078d7ea6 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:16.338472 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f24737-c626-4571-bbd6-1ce3078d7ea6 None admin] PATH_INFO: `/v3/groups` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:16.339195 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f24737-c626-4571-bbd6-1ce3078d7ea6 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:16.340659 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f24737-c626-4571-bbd6-1ce3078d7ea6 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:16.376704 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4229227e-342f-45fe-b982-70a2354968b0 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:16.414585 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f0b3b3-afeb-4f66-9447-f54b7e6d5e95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:16.415523 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f0b3b3-afeb-4f66-9447-f54b7e6d5e95 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:16.415811 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f0b3b3-afeb-4f66-9447-f54b7e6d5e95 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:16.416111 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f0b3b3-afeb-4f66-9447-f54b7e6d5e95 None admin] PATH_INFO: `/v3/users/ebc26aa84a784c3c9ccd166f1096547b` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:16.423259 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 155/303] 10.222.0.44 () {60 vars in 1134 bytes} [Wed Jun 29 11:06:16 2022] POST /identity/v3/groups => generated 229 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:06:16.425214 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfe66e1-deaa-4f73-b818-a25c8ca4bf6e None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:16.426042 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfe66e1-deaa-4f73-b818-a25c8ca4bf6e None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:16.426455 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfe66e1-deaa-4f73-b818-a25c8ca4bf6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:16.432018 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f0b3b3-afeb-4f66-9447-f54b7e6d5e95 None admin] RBAC: Authorizing `identity:get_user(user_id=ebc26aa84a784c3c9ccd166f1096547b)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:16.434045 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f0b3b3-afeb-4f66-9447-f54b7e6d5e95 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:16.442405 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 149/304] 10.222.0.44 () {56 vars in 1177 bytes} [Wed Jun 29 11:06:16 2022] GET /identity/v3/users/ebc26aa84a784c3c9ccd166f1096547b => generated 293 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 11:06:16.444403 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f0b3b3-afeb-4f66-9447-f54b7e6d5e95 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:16.449441 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-5bfe66e1-deaa-4f73-b818-a25c8ca4bf6e None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:06:16.450175 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-5bfe66e1-deaa-4f73-b818-a25c8ca4bf6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:06:16.537877 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:16.547422 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 156/305] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:06:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 11:06:16.561445 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f0b3b3-afeb-4f66-9447-f54b7e6d5e95 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:16.571413 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfe66e1-deaa-4f73-b818-a25c8ca4bf6e None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:16.603151 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f776fe-c2d7-4476-ab5f-8419e41354d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:16.603738 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f776fe-c2d7-4476-ab5f-8419e41354d0 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:16.604051 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f776fe-c2d7-4476-ab5f-8419e41354d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:16.604329 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f776fe-c2d7-4476-ab5f-8419e41354d0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:16.605158 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f776fe-c2d7-4476-ab5f-8419e41354d0 None admin] RBAC: Adding query filter params (role.id=7416219b213f48a4a93d964e67f990fe, user.id=ebc26aa84a784c3c9ccd166f1096547b, scope.system=all) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:06:16.605468 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f776fe-c2d7-4476-ab5f-8419e41354d0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:16.606895 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f776fe-c2d7-4476-ab5f-8419e41354d0 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:16.612459 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 150/306] 10.222.0.44 () {56 vars in 1308 bytes} [Wed Jun 29 11:06:16 2022] GET /identity/v3/role_assignments?user.id=ebc26aa84a784c3c9ccd166f1096547b&scope.system=all&role.id=7416219b213f48a4a93d964e67f990fe => generated 226 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:16.693399 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfe66e1-deaa-4f73-b818-a25c8ca4bf6e None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:16.718039 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:16.741014 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878e96d0-25e6-42da-a1d5-24c6e5299382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:16.742912 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878e96d0-25e6-42da-a1d5-24c6e5299382 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:16.742912 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878e96d0-25e6-42da-a1d5-24c6e5299382 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:16.742912 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878e96d0-25e6-42da-a1d5-24c6e5299382 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:16.754383 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878e96d0-25e6-42da-a1d5-24c6e5299382 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 11:06:16.763211 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-878e96d0-25e6-42da-a1d5-24c6e5299382 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 29 11:06:16.763211 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:06:16.763211 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:06:16.763211 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:06:16.763211 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:06:16.763211 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:06:16.763211 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:06:16.763211 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:06:16.763211 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:06:16.763211 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:06:16.763211 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:06:16.763211 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:06:16.763211 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 11:06:16.763211 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 11:06:16.763211 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 11:06:16.763211 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 11:06:16.764224 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 11:06:16.764224 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 11:06:16.764224 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 11:06:16.764224 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 11:06:16.764224 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:06:16.764224 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:06:16.764224 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:06:16.764224 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:06:16.764224 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:06:16.764224 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:06:16.764224 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:06:16.764224 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jun 29 11:06:16.764224 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:06:16.764224 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:06:16.764224 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:06:16.764224 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:06:16.765104 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:06:16.765104 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:06:16.765104 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:06:16.765104 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:06:16.765104 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 11:06:16.765104 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 11:06:16.765104 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 11:06:16.765104 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 11:06:16.765104 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 11:06:16.765104 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 11:06:16.765104 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 29 11:06:16.765104 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jun 29 11:06:16.769685 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 157/307] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Jun 29 11:06:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 11:06:16.770971 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f776fe-c2d7-4476-ab5f-8419e41354d0 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:16.876477 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f776fe-c2d7-4476-ab5f-8419e41354d0 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:16.912298 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cd6e74-cbe9-4a66-8b86-e7455b38790a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:16.913535 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cd6e74-cbe9-4a66-8b86-e7455b38790a None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:16.913817 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cd6e74-cbe9-4a66-8b86-e7455b38790a None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:16.914132 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cd6e74-cbe9-4a66-8b86-e7455b38790a None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:16.914886 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cd6e74-cbe9-4a66-8b86-e7455b38790a None admin] RBAC: Adding query filter params (name=member) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:06:16.915230 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cd6e74-cbe9-4a66-8b86-e7455b38790a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:16.916681 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cd6e74-cbe9-4a66-8b86-e7455b38790a None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:16.925335 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 151/308] 10.222.0.44 () {56 vars in 1101 bytes} [Wed Jun 29 11:06:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:16.931089 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878e96d0-25e6-42da-a1d5-24c6e5299382 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:17.041244 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878e96d0-25e6-42da-a1d5-24c6e5299382 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:17.075646 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890af3a4-7445-4384-9447-241e78a8c524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:17.076538 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890af3a4-7445-4384-9447-241e78a8c524 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:17.076816 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890af3a4-7445-4384-9447-241e78a8c524 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:17.077084 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890af3a4-7445-4384-9447-241e78a8c524 None admin] PATH_INFO: `/v3/users/1d6218cc192148cfb5604f670e863c20` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:17.078272 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890af3a4-7445-4384-9447-241e78a8c524 None admin] RBAC: Authorizing `identity:get_user(user_id=1d6218cc192148cfb5604f670e863c20)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:17.079799 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890af3a4-7445-4384-9447-241e78a8c524 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:17.087011 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 158/309] 10.222.0.44 () {56 vars in 1177 bytes} [Wed Jun 29 11:06:16 2022] GET /identity/v3/users/1d6218cc192148cfb5604f670e863c20 => generated 293 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:17.088383 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cd6e74-cbe9-4a66-8b86-e7455b38790a None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:17.092346 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:17.196738 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cd6e74-cbe9-4a66-8b86-e7455b38790a None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:17.233261 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1347351f-7ba4-4571-be89-8256e8363732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:17.234496 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1347351f-7ba4-4571-be89-8256e8363732 None admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:17.235002 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1347351f-7ba4-4571-be89-8256e8363732 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:17.235476 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1347351f-7ba4-4571-be89-8256e8363732 None admin] PATH_INFO: `/v3/system/users/1d6218cc192148cfb5604f670e863c20/roles/e1c88eebe911432cab6a07fbeb84653b` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:17.237268 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1347351f-7ba4-4571-be89-8256e8363732 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=1d6218cc192148cfb5604f670e863c20, role_id=e1c88eebe911432cab6a07fbeb84653b)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:17.238868 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1347351f-7ba4-4571-be89-8256e8363732 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:17.337674 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 152/310] 10.222.0.44 () {58 vars in 1334 bytes} [Wed Jun 29 11:06:17 2022] PUT /identity/v3/system/users/1d6218cc192148cfb5604f670e863c20/roles/e1c88eebe911432cab6a07fbeb84653b => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:06:17.432342 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db0c758-a8ae-47b0-9ce4-b29b3633e09e None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:17.432674 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db0c758-a8ae-47b0-9ce4-b29b3633e09e None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:17.432906 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db0c758-a8ae-47b0-9ce4-b29b3633e09e None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:17.434436 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 159/311] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:06:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:06:17.440315 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb28afa4-4601-43f9-aa82-8a82358d62cb None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:17.440608 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb28afa4-4601-43f9-aa82-8a82358d62cb None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:17.440893 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb28afa4-4601-43f9-aa82-8a82358d62cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:17.460570 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-fb28afa4-4601-43f9-aa82-8a82358d62cb None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:06:17.461861 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-fb28afa4-4601-43f9-aa82-8a82358d62cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:06:17.529760 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:17.535527 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 153/312] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:06:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 11:06:17.560862 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db0c758-a8ae-47b0-9ce4-b29b3633e09e None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:17.606448 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:17.676156 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db0c758-a8ae-47b0-9ce4-b29b3633e09e None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:17.711450 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d913ad98-0b92-419e-b556-f5cc364e114c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:17.712180 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d913ad98-0b92-419e-b556-f5cc364e114c None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:17.712459 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d913ad98-0b92-419e-b556-f5cc364e114c None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:17.712730 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d913ad98-0b92-419e-b556-f5cc364e114c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:17.723440 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d913ad98-0b92-419e-b556-f5cc364e114c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 11:06:17.731774 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-d913ad98-0b92-419e-b556-f5cc364e114c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 29 11:06:17.731774 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:06:17.731774 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:06:17.731774 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:06:17.731774 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:06:17.731774 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:06:17.731774 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:06:17.731774 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:06:17.731774 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:06:17.731774 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:06:17.731774 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:06:17.731774 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:06:17.731774 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 11:06:17.731774 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 11:06:17.731774 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 11:06:17.731774 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 11:06:17.732674 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 11:06:17.732674 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 11:06:17.732674 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 11:06:17.732674 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 11:06:17.732674 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:06:17.732674 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:06:17.732674 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:06:17.732674 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:06:17.732674 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:06:17.732674 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:06:17.732674 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:06:17.732674 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jun 29 11:06:17.732674 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:06:17.732674 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:06:17.732674 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:06:17.732674 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:06:17.733473 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:06:17.733473 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:06:17.733473 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:06:17.733473 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:06:17.733473 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 11:06:17.733473 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 11:06:17.733473 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 11:06:17.733473 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 11:06:17.733473 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 11:06:17.733473 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 11:06:17.733473 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 29 11:06:17.733473 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jun 29 11:06:17.733473 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 160/313] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Jun 29 11:06:17 2022] GET /identity/v3/roles/reader => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:06:17.744902 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb28afa4-4601-43f9-aa82-8a82358d62cb None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:17.856481 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb28afa4-4601-43f9-aa82-8a82358d62cb None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:17.890937 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422ebeaf-6787-40db-856f-42062f260923 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:17.891638 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422ebeaf-6787-40db-856f-42062f260923 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:17.891907 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422ebeaf-6787-40db-856f-42062f260923 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:17.892182 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422ebeaf-6787-40db-856f-42062f260923 None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:17.892919 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422ebeaf-6787-40db-856f-42062f260923 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:06:17.893222 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422ebeaf-6787-40db-856f-42062f260923 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:17.894593 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422ebeaf-6787-40db-856f-42062f260923 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:17.907344 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 154/314] 10.222.0.44 () {56 vars in 1101 bytes} [Wed Jun 29 11:06:17 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:17.913458 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d913ad98-0b92-419e-b556-f5cc364e114c None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:18.017911 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:18.021081 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d913ad98-0b92-419e-b556-f5cc364e114c None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:18.055417 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3bccca-6b62-4ac3-9941-e52fe000b25e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:18.056314 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3bccca-6b62-4ac3-9941-e52fe000b25e None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:18.056596 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3bccca-6b62-4ac3-9941-e52fe000b25e None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:18.056868 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3bccca-6b62-4ac3-9941-e52fe000b25e None admin] PATH_INFO: `/v3/users/9e991dc7aae347a08469270f3213700f` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:18.058056 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3bccca-6b62-4ac3-9941-e52fe000b25e None admin] RBAC: Authorizing `identity:get_user(user_id=9e991dc7aae347a08469270f3213700f)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:18.059524 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3bccca-6b62-4ac3-9941-e52fe000b25e None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:18.061628 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 161/315] 10.222.0.44 () {56 vars in 1177 bytes} [Wed Jun 29 11:06:17 2022] GET /identity/v3/users/9e991dc7aae347a08469270f3213700f => generated 297 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:18.066856 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422ebeaf-6787-40db-856f-42062f260923 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:18.129716 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:18.173670 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422ebeaf-6787-40db-856f-42062f260923 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:18.210562 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27a9d27-38bd-46c8-ae5a-00f88f03f48f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:18.211215 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27a9d27-38bd-46c8-ae5a-00f88f03f48f None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:18.211502 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27a9d27-38bd-46c8-ae5a-00f88f03f48f None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:18.211778 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27a9d27-38bd-46c8-ae5a-00f88f03f48f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:18.283489 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27a9d27-38bd-46c8-ae5a-00f88f03f48f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:18.284908 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27a9d27-38bd-46c8-ae5a-00f88f03f48f None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:18.346743 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:18.390628 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 155/316] 10.222.0.44 () {58 vars in 1282 bytes} [Wed Jun 29 11:06:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 29 11:06:18.396291 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3bccca-6b62-4ac3-9941-e52fe000b25e None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:18.498868 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3bccca-6b62-4ac3-9941-e52fe000b25e None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:18.535261 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512d923a-9eeb-4e4c-8073-93236d4a51ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:18.536075 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512d923a-9eeb-4e4c-8073-93236d4a51ea None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:18.536363 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512d923a-9eeb-4e4c-8073-93236d4a51ea None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:18.536639 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512d923a-9eeb-4e4c-8073-93236d4a51ea None admin] PATH_INFO: `/v3/projects/d36c6c9eb3234e82bb6a8537851eb336` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:18.537926 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512d923a-9eeb-4e4c-8073-93236d4a51ea None admin] RBAC: Authorizing `identity:get_project(project_id=d36c6c9eb3234e82bb6a8537851eb336)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:18.539570 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512d923a-9eeb-4e4c-8073-93236d4a51ea None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:18.541609 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 162/317] 10.222.0.44 () {56 vars in 1186 bytes} [Wed Jun 29 11:06:18 2022] GET /identity/v3/projects/d36c6c9eb3234e82bb6a8537851eb336 => generated 302 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:18.547207 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27a9d27-38bd-46c8-ae5a-00f88f03f48f None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:18.649905 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e850ffa-39df-4f82-9f2e-1899d7546577 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:18.650247 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e850ffa-39df-4f82-9f2e-1899d7546577 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:18.650527 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e850ffa-39df-4f82-9f2e-1899d7546577 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:18.652055 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 163/318] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:06:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:06:18.655654 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27a9d27-38bd-46c8-ae5a-00f88f03f48f None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:18.657107 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6c372d-54f9-4a65-83ff-c1db64b3fbba None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:18.657397 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6c372d-54f9-4a65-83ff-c1db64b3fbba None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:18.657678 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6c372d-54f9-4a65-83ff-c1db64b3fbba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:18.678636 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-0f6c372d-54f9-4a65-83ff-c1db64b3fbba None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:06:18.679864 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-0f6c372d-54f9-4a65-83ff-c1db64b3fbba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:06:18.692078 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c26608-6128-46d5-a1ca-fef5e6d8fcf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:18.692740 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c26608-6128-46d5-a1ca-fef5e6d8fcf5 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:18.693026 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c26608-6128-46d5-a1ca-fef5e6d8fcf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:18.693301 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c26608-6128-46d5-a1ca-fef5e6d8fcf5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:18.694695 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c26608-6128-46d5-a1ca-fef5e6d8fcf5 None admin] RBAC: Adding query filter params (role.id=7416219b213f48a4a93d964e67f990fe, scope.project.id=d36c6c9eb3234e82bb6a8537851eb336, user.id=9e991dc7aae347a08469270f3213700f) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:06:18.695318 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c26608-6128-46d5-a1ca-fef5e6d8fcf5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:18.696777 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c26608-6128-46d5-a1ca-fef5e6d8fcf5 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:18.703375 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 156/319] 10.222.0.44 () {56 vars in 1374 bytes} [Wed Jun 29 11:06:18 2022] GET /identity/v3/role_assignments?user.id=9e991dc7aae347a08469270f3213700f&scope.project.id=d36c6c9eb3234e82bb6a8537851eb336&role.id=7416219b213f48a4a93d964e67f990fe => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:18.712272 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:18.753322 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 164/320] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:06:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 11:06:18.778651 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c26608-6128-46d5-a1ca-fef5e6d8fcf5 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:18.807348 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:18.819332 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283f139e-6794-45a0-9aa7-a2c1a3c85f0f None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:18.819694 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283f139e-6794-45a0-9aa7-a2c1a3c85f0f None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:18.820008 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283f139e-6794-45a0-9aa7-a2c1a3c85f0f None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:18.821620 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 165/321] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:06:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:06:18.828894 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8aa080-f275-4b4c-8d6b-18d7868ac2b4 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:18.829181 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8aa080-f275-4b4c-8d6b-18d7868ac2b4 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:18.829441 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8aa080-f275-4b4c-8d6b-18d7868ac2b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:18.853682 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-6f8aa080-f275-4b4c-8d6b-18d7868ac2b4 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:06:18.855208 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-6f8aa080-f275-4b4c-8d6b-18d7868ac2b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:06:18.903595 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c26608-6128-46d5-a1ca-fef5e6d8fcf5 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:18.944306 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6912077-b5c4-41d7-b566-ee73ea1873ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:18.945102 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6912077-b5c4-41d7-b566-ee73ea1873ab None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:18.945408 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6912077-b5c4-41d7-b566-ee73ea1873ab None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:18.945661 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6912077-b5c4-41d7-b566-ee73ea1873ab None admin] PATH_INFO: `/v3/domains/default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:18.947044 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6912077-b5c4-41d7-b566-ee73ea1873ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:18.948586 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6912077-b5c4-41d7-b566-ee73ea1873ab None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:18.951282 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 157/322] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jun 29 11:06:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:18.956265 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6912077-b5c4-41d7-b566-ee73ea1873ab None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:18.957381 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 166/323] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:06:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 29 11:06:18.984232 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8aa080-f275-4b4c-8d6b-18d7868ac2b4 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:19.060947 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:19.072596 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6912077-b5c4-41d7-b566-ee73ea1873ab None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:19.087468 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:19.092938 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8aa080-f275-4b4c-8d6b-18d7868ac2b4 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:19.108757 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8643cee1-277b-45b5-96b7-98c1aad9726c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:19.109425 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8643cee1-277b-45b5-96b7-98c1aad9726c None admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:19.109712 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8643cee1-277b-45b5-96b7-98c1aad9726c None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:19.109989 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8643cee1-277b-45b5-96b7-98c1aad9726c None admin] PATH_INFO: `/v3/groups` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:19.110689 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8643cee1-277b-45b5-96b7-98c1aad9726c None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:19.112132 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8643cee1-277b-45b5-96b7-98c1aad9726c None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:19.127526 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11556a0-ec90-4c08-93e7-3d6ec1172675 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:19.128537 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11556a0-ec90-4c08-93e7-3d6ec1172675 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:19.128813 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11556a0-ec90-4c08-93e7-3d6ec1172675 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:19.129144 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11556a0-ec90-4c08-93e7-3d6ec1172675 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:19.140181 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11556a0-ec90-4c08-93e7-3d6ec1172675 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 11:06:19.147960 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-a11556a0-ec90-4c08-93e7-3d6ec1172675 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 29 11:06:19.147960 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:06:19.147960 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:06:19.147960 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:06:19.147960 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:06:19.147960 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:06:19.147960 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:06:19.147960 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:06:19.147960 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:06:19.147960 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:06:19.147960 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:06:19.147960 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:06:19.147960 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 11:06:19.147960 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 11:06:19.147960 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 11:06:19.147960 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 11:06:19.149025 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 11:06:19.149025 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 11:06:19.149025 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 11:06:19.149025 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 11:06:19.149025 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:06:19.149025 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:06:19.149025 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:06:19.149025 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:06:19.149025 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:06:19.149025 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:06:19.149025 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:06:19.149025 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jun 29 11:06:19.149025 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:06:19.149025 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:06:19.149025 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:06:19.149025 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:06:19.149951 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:06:19.149951 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:06:19.149951 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:06:19.149951 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:06:19.149951 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 11:06:19.149951 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 11:06:19.149951 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 11:06:19.149951 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 11:06:19.149951 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 11:06:19.149951 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 11:06:19.149951 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 29 11:06:19.149951 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jun 29 11:06:19.149951 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 167/324] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Jun 29 11:06:18 2022] GET /identity/v3/roles/reader => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 11:06:19.154921 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11556a0-ec90-4c08-93e7-3d6ec1172675 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:19.200839 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 158/325] 10.222.0.44 () {60 vars in 1134 bytes} [Wed Jun 29 11:06:18 2022] POST /identity/v3/groups => generated 226 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 11:06:19.267114 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11556a0-ec90-4c08-93e7-3d6ec1172675 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:19.302584 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027f6fbd-4d96-432d-856e-eda56c6f8133 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:19.303771 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027f6fbd-4d96-432d-856e-eda56c6f8133 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:19.304087 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027f6fbd-4d96-432d-856e-eda56c6f8133 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:19.304363 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027f6fbd-4d96-432d-856e-eda56c6f8133 None admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:19.305127 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027f6fbd-4d96-432d-856e-eda56c6f8133 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:06:19.305433 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027f6fbd-4d96-432d-856e-eda56c6f8133 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:19.306840 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027f6fbd-4d96-432d-856e-eda56c6f8133 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:19.316499 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 168/326] 10.222.0.44 () {56 vars in 1101 bytes} [Wed Jun 29 11:06:19 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 11:06:19.322735 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8643cee1-277b-45b5-96b7-98c1aad9726c None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:19.428763 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8643cee1-277b-45b5-96b7-98c1aad9726c None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:19.464734 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1582996f-1086-4505-88a2-25ba953caaef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:19.465867 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1582996f-1086-4505-88a2-25ba953caaef None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:19.466154 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1582996f-1086-4505-88a2-25ba953caaef None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:19.466433 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1582996f-1086-4505-88a2-25ba953caaef None admin] PATH_INFO: `/v3/users/ebc26aa84a784c3c9ccd166f1096547b` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:19.467649 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1582996f-1086-4505-88a2-25ba953caaef None admin] RBAC: Authorizing `identity:get_user(user_id=ebc26aa84a784c3c9ccd166f1096547b)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:19.469141 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1582996f-1086-4505-88a2-25ba953caaef None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:19.471046 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 159/327] 10.222.0.44 () {56 vars in 1177 bytes} [Wed Jun 29 11:06:19 2022] GET /identity/v3/users/ebc26aa84a784c3c9ccd166f1096547b => generated 293 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:19.477025 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027f6fbd-4d96-432d-856e-eda56c6f8133 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:19.494044 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c35d55-b01d-4ba4-83c5-4a291faa8617 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:19.494333 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c35d55-b01d-4ba4-83c5-4a291faa8617 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:19.494608 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c35d55-b01d-4ba4-83c5-4a291faa8617 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:19.496123 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 160/328] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:06:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 29 11:06:19.499990 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:19.500959 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055a788a-81c2-47ff-855b-77cec28bcf5c None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:19.501253 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055a788a-81c2-47ff-855b-77cec28bcf5c None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:19.501534 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055a788a-81c2-47ff-855b-77cec28bcf5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:19.520597 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-055a788a-81c2-47ff-855b-77cec28bcf5c None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:06:19.521222 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-055a788a-81c2-47ff-855b-77cec28bcf5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:06:19.538773 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:19.603364 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027f6fbd-4d96-432d-856e-eda56c6f8133 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:19.609346 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 161/329] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:06:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 29 11:06:19.632884 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055a788a-81c2-47ff-855b-77cec28bcf5c None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:19.647092 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e569b2-5581-43f3-9778-a5559ab7db40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:19.648476 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e569b2-5581-43f3-9778-a5559ab7db40 None admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:19.648766 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e569b2-5581-43f3-9778-a5559ab7db40 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:19.649324 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e569b2-5581-43f3-9778-a5559ab7db40 None admin] PATH_INFO: `/v3/system/users/ebc26aa84a784c3c9ccd166f1096547b/roles/7416219b213f48a4a93d964e67f990fe` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:19.651114 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e569b2-5581-43f3-9778-a5559ab7db40 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=ebc26aa84a784c3c9ccd166f1096547b, role_id=7416219b213f48a4a93d964e67f990fe)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:19.652944 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e569b2-5581-43f3-9778-a5559ab7db40 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:19.725416 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 169/330] 10.222.0.44 () {58 vars in 1334 bytes} [Wed Jun 29 11:06:19 2022] PUT /identity/v3/system/users/ebc26aa84a784c3c9ccd166f1096547b/roles/7416219b213f48a4a93d964e67f990fe => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 11:06:19.757134 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055a788a-81c2-47ff-855b-77cec28bcf5c None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:19.796598 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bebbd11-7c86-4bff-9fe0-e496053cad4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:19.797414 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bebbd11-7c86-4bff-9fe0-e496053cad4c None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:19.797686 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bebbd11-7c86-4bff-9fe0-e496053cad4c None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:19.797958 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bebbd11-7c86-4bff-9fe0-e496053cad4c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:19.810967 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bebbd11-7c86-4bff-9fe0-e496053cad4c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 11:06:19.819710 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-4bebbd11-7c86-4bff-9fe0-e496053cad4c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 29 11:06:19.819710 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:06:19.819710 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:06:19.819710 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:06:19.819710 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:06:19.819710 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:06:19.819710 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:06:19.819710 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:06:19.819710 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:06:19.819710 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:06:19.819710 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:06:19.819710 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:06:19.819710 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 11:06:19.819710 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 11:06:19.819710 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 11:06:19.819710 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 11:06:19.820662 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 11:06:19.820662 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 11:06:19.820662 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 11:06:19.820662 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 11:06:19.820662 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:06:19.820662 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:06:19.820662 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:06:19.820662 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:06:19.820662 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:06:19.820662 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:06:19.820662 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:06:19.820662 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jun 29 11:06:19.820662 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:06:19.820662 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:06:19.820662 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:06:19.820662 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:06:19.822199 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:06:19.822199 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:06:19.822199 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:06:19.822199 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:06:19.822199 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 11:06:19.822199 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 11:06:19.822199 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 11:06:19.822199 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 11:06:19.822199 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 11:06:19.822199 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 11:06:19.822199 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 29 11:06:19.822199 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:06:19.822199 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 162/331] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Jun 29 11:06:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 11:06:19.827803 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e569b2-5581-43f3-9778-a5559ab7db40 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:19.935608 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:19.955853 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e569b2-5581-43f3-9778-a5559ab7db40 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:19.992659 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c35bb73-1ca4-47bf-844b-0a56be5d1747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:19.993440 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c35bb73-1ca4-47bf-844b-0a56be5d1747 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:19.993727 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c35bb73-1ca4-47bf-844b-0a56be5d1747 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:19.994006 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c35bb73-1ca4-47bf-844b-0a56be5d1747 None admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:19.994808 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c35bb73-1ca4-47bf-844b-0a56be5d1747 None admin] RBAC: Adding query filter params (name=member) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:06:19.995113 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c35bb73-1ca4-47bf-844b-0a56be5d1747 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:19.996585 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c35bb73-1ca4-47bf-844b-0a56be5d1747 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:20.007907 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 170/332] 10.222.0.44 () {56 vars in 1101 bytes} [Wed Jun 29 11:06:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:20.013030 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bebbd11-7c86-4bff-9fe0-e496053cad4c None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:20.028713 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:20.137294 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bebbd11-7c86-4bff-9fe0-e496053cad4c None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:20.176863 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57a1583-9068-49d0-8f81-7e5944a24d88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:20.177668 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57a1583-9068-49d0-8f81-7e5944a24d88 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:20.178520 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57a1583-9068-49d0-8f81-7e5944a24d88 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:20.178800 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57a1583-9068-49d0-8f81-7e5944a24d88 None admin] PATH_INFO: `/v3/users/1d6218cc192148cfb5604f670e863c20` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:20.182103 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57a1583-9068-49d0-8f81-7e5944a24d88 None admin] RBAC: Authorizing `identity:get_user(user_id=1d6218cc192148cfb5604f670e863c20)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:20.183577 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57a1583-9068-49d0-8f81-7e5944a24d88 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:20.185622 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 163/333] 10.222.0.44 () {56 vars in 1177 bytes} [Wed Jun 29 11:06:20 2022] GET /identity/v3/users/1d6218cc192148cfb5604f670e863c20 => generated 293 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:20.191342 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c35bb73-1ca4-47bf-844b-0a56be5d1747 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:20.299030 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c35bb73-1ca4-47bf-844b-0a56be5d1747 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:20.335409 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69ce890-3a47-44be-9b62-48c431ce1633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:20.336096 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69ce890-3a47-44be-9b62-48c431ce1633 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:20.336382 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69ce890-3a47-44be-9b62-48c431ce1633 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:20.336655 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69ce890-3a47-44be-9b62-48c431ce1633 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:20.337506 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69ce890-3a47-44be-9b62-48c431ce1633 None admin] RBAC: Adding query filter params (role.id=e1c88eebe911432cab6a07fbeb84653b, user.id=1d6218cc192148cfb5604f670e863c20, scope.system=all) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:06:20.337817 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69ce890-3a47-44be-9b62-48c431ce1633 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:20.339273 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69ce890-3a47-44be-9b62-48c431ce1633 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:20.352461 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 171/334] 10.222.0.44 () {56 vars in 1308 bytes} [Wed Jun 29 11:06:20 2022] GET /identity/v3/role_assignments?user.id=1d6218cc192148cfb5604f670e863c20&scope.system=all&role.id=e1c88eebe911432cab6a07fbeb84653b => generated 510 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:21.671007 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea82ebf-9a7a-48f2-84c1-b0d289225263 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:21.672061 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea82ebf-9a7a-48f2-84c1-b0d289225263 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:21.672548 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea82ebf-9a7a-48f2-84c1-b0d289225263 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:21.674358 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 164/335] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:06:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:06:21.680528 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e11b0a-f5fb-41f1-abd7-bb908e5fb403 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:21.681448 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e11b0a-f5fb-41f1-abd7-bb908e5fb403 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:21.681922 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e11b0a-f5fb-41f1-abd7-bb908e5fb403 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:21.687657 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:21.703303 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-48e11b0a-f5fb-41f1-abd7-bb908e5fb403 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:06:21.704845 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-48e11b0a-f5fb-41f1-abd7-bb908e5fb403 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:06:21.798138 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 172/336] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:06:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 11:06:21.833649 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea82ebf-9a7a-48f2-84c1-b0d289225263 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:21.959383 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:21.982329 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea82ebf-9a7a-48f2-84c1-b0d289225263 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:22.022288 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0160311-855d-477c-abcb-058245c2103c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:22.023092 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0160311-855d-477c-abcb-058245c2103c None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:22.023375 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0160311-855d-477c-abcb-058245c2103c None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:22.023650 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0160311-855d-477c-abcb-058245c2103c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:22.041170 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0160311-855d-477c-abcb-058245c2103c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 11:06:22.052475 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-a0160311-855d-477c-abcb-058245c2103c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 29 11:06:22.052475 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:06:22.052475 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:06:22.052475 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:06:22.052475 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:06:22.052475 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:06:22.052475 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:06:22.052475 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:06:22.052475 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:06:22.052475 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:06:22.052475 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:06:22.052475 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:06:22.052475 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 11:06:22.052475 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 11:06:22.052475 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 11:06:22.052475 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 11:06:22.053961 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 11:06:22.053961 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 11:06:22.053961 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 11:06:22.053961 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 11:06:22.053961 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:06:22.053961 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:06:22.053961 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:06:22.053961 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:06:22.053961 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:06:22.053961 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:06:22.053961 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:06:22.053961 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jun 29 11:06:22.053961 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:06:22.053961 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:06:22.053961 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:06:22.053961 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:06:22.054764 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:06:22.054764 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:06:22.054764 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:06:22.054764 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:06:22.054764 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 11:06:22.054764 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 11:06:22.054764 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 11:06:22.054764 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 11:06:22.054764 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 11:06:22.054764 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 11:06:22.054764 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 29 11:06:22.054764 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:06:22.054764 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 165/337] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Jun 29 11:06:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:06:22.061506 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e11b0a-f5fb-41f1-abd7-bb908e5fb403 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:22.185665 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e11b0a-f5fb-41f1-abd7-bb908e5fb403 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:22.221528 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:22.230936 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c063387f-71e7-449c-bb52-9f61fd511c5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:22.236346 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c063387f-71e7-449c-bb52-9f61fd511c5c None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:22.236346 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c063387f-71e7-449c-bb52-9f61fd511c5c None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:22.236346 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c063387f-71e7-449c-bb52-9f61fd511c5c None admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:22.236346 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c063387f-71e7-449c-bb52-9f61fd511c5c None admin] RBAC: Adding query filter params (name=member) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:06:22.236346 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c063387f-71e7-449c-bb52-9f61fd511c5c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:22.236346 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c063387f-71e7-449c-bb52-9f61fd511c5c None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:22.244433 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 173/338] 10.222.0.44 () {56 vars in 1101 bytes} [Wed Jun 29 11:06:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:22.250011 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0160311-855d-477c-abcb-058245c2103c None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:22.373238 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0160311-855d-477c-abcb-058245c2103c None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:22.408167 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64fc0a5-562c-4fe6-9fef-744003125941 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:22.408516 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64fc0a5-562c-4fe6-9fef-744003125941 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:22.408724 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64fc0a5-562c-4fe6-9fef-744003125941 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:22.410394 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 174/339] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:06:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 29 11:06:22.421725 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b02996a-3345-4938-a42a-797bc1ca9c0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:22.422385 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b02996a-3345-4938-a42a-797bc1ca9c0c None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:22.422750 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b02996a-3345-4938-a42a-797bc1ca9c0c None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:22.423032 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b02996a-3345-4938-a42a-797bc1ca9c0c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:22.425439 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2317a9-d160-4e5f-b89d-09359cdcc96d None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:22.425728 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2317a9-d160-4e5f-b89d-09359cdcc96d None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:22.426032 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2317a9-d160-4e5f-b89d-09359cdcc96d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:22.440225 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:22.464862 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-1e2317a9-d160-4e5f-b89d-09359cdcc96d None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:06:22.466081 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-1e2317a9-d160-4e5f-b89d-09359cdcc96d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:06:22.536006 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b02996a-3345-4938-a42a-797bc1ca9c0c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:22.537943 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b02996a-3345-4938-a42a-797bc1ca9c0c None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:22.616047 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 175/340] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:06:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 29 11:06:22.617391 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b900b43-6924-40e5-9bca-6a0f692ae5fa None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:22.617736 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b900b43-6924-40e5-9bca-6a0f692ae5fa None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:22.618020 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b900b43-6924-40e5-9bca-6a0f692ae5fa None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:22.624428 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 176/341] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:06:22 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:06:22.625608 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcbeda9-d5f2-4f77-9488-343259fba407 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:22.625889 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcbeda9-d5f2-4f77-9488-343259fba407 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:22.626163 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcbeda9-d5f2-4f77-9488-343259fba407 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:22.658268 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-cbcbeda9-d5f2-4f77-9488-343259fba407 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:06:22.659440 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-cbcbeda9-d5f2-4f77-9488-343259fba407 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:06:22.755343 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 166/342] 10.222.0.44 () {58 vars in 1282 bytes} [Wed Jun 29 11:06:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 505 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 29 11:06:22.757427 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b02996a-3345-4938-a42a-797bc1ca9c0c None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:22.793394 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 177/343] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:06:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 11:06:22.796686 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcbeda9-d5f2-4f77-9488-343259fba407 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:22.833595 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:22.880695 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:22.941523 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b02996a-3345-4938-a42a-797bc1ca9c0c None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:22.968792 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcbeda9-d5f2-4f77-9488-343259fba407 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:22.985492 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59aaa12-eab1-489c-a020-4a6625a5161b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:22.986320 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59aaa12-eab1-489c-a020-4a6625a5161b None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:22.986604 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59aaa12-eab1-489c-a020-4a6625a5161b None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:22.986879 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59aaa12-eab1-489c-a020-4a6625a5161b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:23.002752 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59aaa12-eab1-489c-a020-4a6625a5161b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 11:06:23.011128 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-b59aaa12-eab1-489c-a020-4a6625a5161b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 29 11:06:23.011128 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:06:23.011128 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:06:23.011128 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:06:23.011128 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:06:23.011128 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:06:23.011128 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:06:23.011128 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:06:23.011128 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:06:23.011128 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:06:23.011128 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:06:23.011128 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:06:23.011128 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 11:06:23.011128 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 11:06:23.011128 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 11:06:23.011128 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 11:06:23.012051 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 11:06:23.012051 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 11:06:23.012051 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 11:06:23.012051 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 11:06:23.012051 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:06:23.012051 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:06:23.012051 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:06:23.012051 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:06:23.012051 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:06:23.012051 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:06:23.012051 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:06:23.012051 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jun 29 11:06:23.012051 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:06:23.012051 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:06:23.012051 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:06:23.012051 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:06:23.012841 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:06:23.012841 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:06:23.012841 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:06:23.012841 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:06:23.012841 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 11:06:23.012841 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 11:06:23.012841 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 11:06:23.012841 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 11:06:23.012841 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 11:06:23.012841 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 11:06:23.012841 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 29 11:06:23.012841 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:06:23.012841 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 167/344] 10.222.0.44 () {56 vars in 1114 bytes} [Wed Jun 29 11:06:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:06:23.017519 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59aaa12-eab1-489c-a020-4a6625a5161b None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:23.023156 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb71700e-4f5f-4c6d-bd36-b1a957341ff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:23.024330 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb71700e-4f5f-4c6d-bd36-b1a957341ff4 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:23.024846 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb71700e-4f5f-4c6d-bd36-b1a957341ff4 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:23.025363 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb71700e-4f5f-4c6d-bd36-b1a957341ff4 None admin] PATH_INFO: `/v3/projects/3b98d22914de4e3b8bca7fe0a0b51357` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:23.026824 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb71700e-4f5f-4c6d-bd36-b1a957341ff4 None admin] RBAC: Authorizing `identity:get_project(project_id=3b98d22914de4e3b8bca7fe0a0b51357)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:23.028619 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb71700e-4f5f-4c6d-bd36-b1a957341ff4 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:23.030900 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 178/345] 10.222.0.44 () {56 vars in 1186 bytes} [Wed Jun 29 11:06:22 2022] GET /identity/v3/projects/3b98d22914de4e3b8bca7fe0a0b51357 => generated 298 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:23.033547 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb71700e-4f5f-4c6d-bd36-b1a957341ff4 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:23.133601 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:23.137581 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59aaa12-eab1-489c-a020-4a6625a5161b None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:23.167262 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb71700e-4f5f-4c6d-bd36-b1a957341ff4 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:23.176968 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ff80ea-0a0b-4ae3-8854-2af6a73ce26d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:23.178621 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ff80ea-0a0b-4ae3-8854-2af6a73ce26d None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:23.180186 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ff80ea-0a0b-4ae3-8854-2af6a73ce26d None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:23.180695 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ff80ea-0a0b-4ae3-8854-2af6a73ce26d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:23.192390 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ff80ea-0a0b-4ae3-8854-2af6a73ce26d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 11:06:23.196105 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:23.202877 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-a5ff80ea-0a0b-4ae3-8854-2af6a73ce26d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 29 11:06:23.202877 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:06:23.202877 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:06:23.202877 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:06:23.202877 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:06:23.202877 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:06:23.202877 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:06:23.202877 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:06:23.202877 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:06:23.202877 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:06:23.202877 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:06:23.202877 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:06:23.202877 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 11:06:23.202877 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 11:06:23.202877 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 11:06:23.202877 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 11:06:23.207754 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 11:06:23.207754 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 11:06:23.207754 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 11:06:23.207754 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 11:06:23.207754 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:06:23.207754 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:06:23.207754 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:06:23.207754 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:06:23.207754 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:06:23.207754 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:06:23.207754 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:06:23.207754 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jun 29 11:06:23.207754 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:06:23.207754 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:06:23.207754 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:06:23.207754 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:06:23.208702 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:06:23.208702 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:06:23.208702 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:06:23.208702 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:06:23.208702 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 11:06:23.208702 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 11:06:23.208702 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 11:06:23.208702 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 11:06:23.208702 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 11:06:23.208702 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 11:06:23.208702 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 29 11:06:23.208702 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:06:23.208702 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 168/346] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Jun 29 11:06:23 2022] GET /identity/v3/roles/reader => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:06:23.208702 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ff80ea-0a0b-4ae3-8854-2af6a73ce26d None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:23.214677 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81358326-f539-4f1f-a09f-89c2de72362e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:23.215418 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81358326-f539-4f1f-a09f-89c2de72362e None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:23.215711 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81358326-f539-4f1f-a09f-89c2de72362e None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:23.216021 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81358326-f539-4f1f-a09f-89c2de72362e None admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:23.216789 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81358326-f539-4f1f-a09f-89c2de72362e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:06:23.217099 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81358326-f539-4f1f-a09f-89c2de72362e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:23.218485 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81358326-f539-4f1f-a09f-89c2de72362e None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:23.233869 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 179/347] 10.222.0.44 () {56 vars in 1111 bytes} [Wed Jun 29 11:06:23 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:23.233869 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81358326-f539-4f1f-a09f-89c2de72362e None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:23.349146 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ff80ea-0a0b-4ae3-8854-2af6a73ce26d None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:23.367484 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81358326-f539-4f1f-a09f-89c2de72362e None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:23.398208 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10eb2b13-3656-424c-ba4d-3a6dffcae88f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:23.400294 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10eb2b13-3656-424c-ba4d-3a6dffcae88f None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:23.402371 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10eb2b13-3656-424c-ba4d-3a6dffcae88f None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:23.402797 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10eb2b13-3656-424c-ba4d-3a6dffcae88f None admin] PATH_INFO: `/v3/groups/8ec7aa0c6a3d47889a0b287a0fc483ad` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:23.417433 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10eb2b13-3656-424c-ba4d-3a6dffcae88f None admin] RBAC: Authorizing `identity:get_group(group_id=8ec7aa0c6a3d47889a0b287a0fc483ad)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:23.419158 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10eb2b13-3656-424c-ba4d-3a6dffcae88f None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:23.422921 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 169/348] 10.222.0.44 () {56 vars in 1180 bytes} [Wed Jun 29 11:06:23 2022] GET /identity/v3/groups/8ec7aa0c6a3d47889a0b287a0fc483ad => generated 226 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:23.426041 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10eb2b13-3656-424c-ba4d-3a6dffcae88f None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:23.445491 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d688e3-40b1-41bc-9fcb-0b013e478b49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:23.446415 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d688e3-40b1-41bc-9fcb-0b013e478b49 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:23.447136 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d688e3-40b1-41bc-9fcb-0b013e478b49 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:23.447559 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d688e3-40b1-41bc-9fcb-0b013e478b49 None admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:23.448487 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d688e3-40b1-41bc-9fcb-0b013e478b49 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:06:23.448942 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d688e3-40b1-41bc-9fcb-0b013e478b49 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:23.450477 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d688e3-40b1-41bc-9fcb-0b013e478b49 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:23.470765 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 180/349] 10.222.0.44 () {56 vars in 1101 bytes} [Wed Jun 29 11:06:23 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:23.476501 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d688e3-40b1-41bc-9fcb-0b013e478b49 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:23.489660 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:23.550708 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:23.611080 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10eb2b13-3656-424c-ba4d-3a6dffcae88f None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:23.614649 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d688e3-40b1-41bc-9fcb-0b013e478b49 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:23.653515 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d654a3-1aeb-4040-9c01-53fbec796b78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:23.655301 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d654a3-1aeb-4040-9c01-53fbec796b78 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:23.655301 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d654a3-1aeb-4040-9c01-53fbec796b78 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:23.655301 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d654a3-1aeb-4040-9c01-53fbec796b78 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:23.655552 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d654a3-1aeb-4040-9c01-53fbec796b78 None admin] RBAC: Adding query filter params (group.id=8ec7aa0c6a3d47889a0b287a0fc483ad, role.id=e1c88eebe911432cab6a07fbeb84653b, scope.project.id=3b98d22914de4e3b8bca7fe0a0b51357) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:06:23.655864 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d654a3-1aeb-4040-9c01-53fbec796b78 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:23.656628 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35b03be-0a0e-479b-a5d7-cbb4b8087c95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:23.657033 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35b03be-0a0e-479b-a5d7-cbb4b8087c95 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:23.657237 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d654a3-1aeb-4040-9c01-53fbec796b78 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:23.657346 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35b03be-0a0e-479b-a5d7-cbb4b8087c95 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:23.658488 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35b03be-0a0e-479b-a5d7-cbb4b8087c95 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:23.666356 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 181/350] 10.222.0.44 () {56 vars in 1376 bytes} [Wed Jun 29 11:06:23 2022] GET /identity/v3/role_assignments?group.id=8ec7aa0c6a3d47889a0b287a0fc483ad&scope.project.id=3b98d22914de4e3b8bca7fe0a0b51357&role.id=e1c88eebe911432cab6a07fbeb84653b => generated 260 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:23.667426 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d654a3-1aeb-4040-9c01-53fbec796b78 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:23.756129 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35b03be-0a0e-479b-a5d7-cbb4b8087c95 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:23.758226 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35b03be-0a0e-479b-a5d7-cbb4b8087c95 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:23.785807 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:23.796682 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d654a3-1aeb-4040-9c01-53fbec796b78 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:23.808955 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:23.837499 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8083a5ab-6d26-431a-a1c8-f39134aa8a36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:23.838828 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8083a5ab-6d26-431a-a1c8-f39134aa8a36 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:23.840078 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8083a5ab-6d26-431a-a1c8-f39134aa8a36 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:23.840388 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8083a5ab-6d26-431a-a1c8-f39134aa8a36 None admin] PATH_INFO: `/v3/users/ebc26aa84a784c3c9ccd166f1096547b` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:23.844724 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8083a5ab-6d26-431a-a1c8-f39134aa8a36 None admin] RBAC: Authorizing `identity:get_user(user_id=ebc26aa84a784c3c9ccd166f1096547b)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:23.847087 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8083a5ab-6d26-431a-a1c8-f39134aa8a36 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:23.850383 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 182/351] 10.222.0.44 () {56 vars in 1177 bytes} [Wed Jun 29 11:06:23 2022] GET /identity/v3/users/ebc26aa84a784c3c9ccd166f1096547b => generated 293 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:23.852356 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfde5ee3-4714-4226-9da4-5a40128ef2ab None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:23.852966 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfde5ee3-4714-4226-9da4-5a40128ef2ab None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:23.853845 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfde5ee3-4714-4226-9da4-5a40128ef2ab None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:23.855640 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 183/352] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:06:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:06:23.857543 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7724872-713e-4865-9b79-817df9bc5d6a None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:23.858011 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7724872-713e-4865-9b79-817df9bc5d6a None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:23.858486 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7724872-713e-4865-9b79-817df9bc5d6a None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:23.860582 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 184/353] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:06:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:06:23.863299 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7724872-713e-4865-9b79-817df9bc5d6a None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:23.883976 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 170/354] 10.222.0.44 () {58 vars in 1282 bytes} [Wed Jun 29 11:06:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 458 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 29 11:06:23.885558 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab29fe0-2057-41fa-b68c-0d2accc52227 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:23.886050 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab29fe0-2057-41fa-b68c-0d2accc52227 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:23.886510 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab29fe0-2057-41fa-b68c-0d2accc52227 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:23.908297 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-8ab29fe0-2057-41fa-b68c-0d2accc52227 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:06:23.909441 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-8ab29fe0-2057-41fa-b68c-0d2accc52227 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:06:24.001819 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7724872-713e-4865-9b79-817df9bc5d6a None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:24.003970 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 171/355] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:06:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 11:06:24.005543 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65a83cd-366e-4b9c-80b3-000b3ff334a6 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:24.006555 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65a83cd-366e-4b9c-80b3-000b3ff334a6 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:24.007039 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65a83cd-366e-4b9c-80b3-000b3ff334a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:24.042312 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-d65a83cd-366e-4b9c-80b3-000b3ff334a6 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:06:24.043407 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d673d824-4228-4f53-8fea-96fc8bc84c52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:24.044107 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d673d824-4228-4f53-8fea-96fc8bc84c52 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:24.044371 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-d65a83cd-366e-4b9c-80b3-000b3ff334a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:06:24.044474 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d673d824-4228-4f53-8fea-96fc8bc84c52 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:24.044718 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d673d824-4228-4f53-8fea-96fc8bc84c52 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:24.045565 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d673d824-4228-4f53-8fea-96fc8bc84c52 None admin] RBAC: Adding query filter params (role.id=7416219b213f48a4a93d964e67f990fe, user.id=ebc26aa84a784c3c9ccd166f1096547b, scope.system=all) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:06:24.045881 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d673d824-4228-4f53-8fea-96fc8bc84c52 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:24.047530 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d673d824-4228-4f53-8fea-96fc8bc84c52 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:24.062769 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 185/356] 10.222.0.44 () {56 vars in 1308 bytes} [Wed Jun 29 11:06:23 2022] GET /identity/v3/role_assignments?user.id=ebc26aa84a784c3c9ccd166f1096547b&scope.system=all&role.id=7416219b213f48a4a93d964e67f990fe => generated 510 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:24.063919 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d673d824-4228-4f53-8fea-96fc8bc84c52 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:24.101878 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:24.131421 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 172/357] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:06:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 11:06:24.132671 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65a83cd-366e-4b9c-80b3-000b3ff334a6 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:24.156353 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:24.210037 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d673d824-4228-4f53-8fea-96fc8bc84c52 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:24.264472 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1d446f-1844-43d3-9627-0d44b43c0a72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:24.265712 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1d446f-1844-43d3-9627-0d44b43c0a72 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:24.266200 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1d446f-1844-43d3-9627-0d44b43c0a72 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:24.267721 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1d446f-1844-43d3-9627-0d44b43c0a72 None admin] PATH_INFO: `/v3/projects/3b98d22914de4e3b8bca7fe0a0b51357` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:24.269372 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1d446f-1844-43d3-9627-0d44b43c0a72 None admin] RBAC: Authorizing `identity:get_project(project_id=3b98d22914de4e3b8bca7fe0a0b51357)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:24.271096 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1d446f-1844-43d3-9627-0d44b43c0a72 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:24.273705 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 186/358] 10.222.0.44 () {56 vars in 1186 bytes} [Wed Jun 29 11:06:24 2022] GET /identity/v3/projects/3b98d22914de4e3b8bca7fe0a0b51357 => generated 298 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:24.275213 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58423966-3a1c-4bbc-909e-12b817d77c9e None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:24.275591 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65a83cd-366e-4b9c-80b3-000b3ff334a6 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:24.281433 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58423966-3a1c-4bbc-909e-12b817d77c9e None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:24.281433 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58423966-3a1c-4bbc-909e-12b817d77c9e None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:24.283321 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 187/359] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:06:24 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:06:24.283321 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58423966-3a1c-4bbc-909e-12b817d77c9e None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:24.323325 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34317792-e1a0-4416-8640-abe480f99b06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:24.324192 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34317792-e1a0-4416-8640-abe480f99b06 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:24.324532 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34317792-e1a0-4416-8640-abe480f99b06 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:24.324803 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34317792-e1a0-4416-8640-abe480f99b06 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:24.338781 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34317792-e1a0-4416-8640-abe480f99b06 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 11:06:24.347073 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-34317792-e1a0-4416-8640-abe480f99b06 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 29 11:06:24.347073 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:06:24.347073 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:06:24.347073 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:06:24.347073 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:06:24.347073 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:06:24.347073 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:06:24.347073 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:06:24.347073 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:06:24.347073 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:06:24.347073 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:06:24.347073 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:06:24.347073 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 11:06:24.347073 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 11:06:24.347073 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 11:06:24.347073 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 11:06:24.348074 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 11:06:24.348074 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 11:06:24.348074 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 11:06:24.348074 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 11:06:24.348074 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:06:24.348074 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:06:24.348074 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:06:24.348074 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:06:24.348074 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:06:24.348074 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:06:24.348074 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:06:24.348074 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jun 29 11:06:24.348074 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:06:24.348074 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:06:24.348074 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:06:24.348074 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:06:24.348866 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:06:24.348866 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:06:24.348866 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:06:24.348866 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:06:24.348866 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 11:06:24.348866 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 11:06:24.348866 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 11:06:24.348866 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 11:06:24.348866 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 11:06:24.348866 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 11:06:24.348866 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 29 11:06:24.348866 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:06:24.349988 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 173/360] 10.222.0.44 () {56 vars in 1114 bytes} [Wed Jun 29 11:06:24 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:06:24.351472 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34317792-e1a0-4416-8640-abe480f99b06 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:24.409627 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:24.423542 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58423966-3a1c-4bbc-909e-12b817d77c9e None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:24.432143 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:24.475571 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b95d52-e243-414a-baa7-334c47693325 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:24.476375 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b95d52-e243-414a-baa7-334c47693325 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:24.477160 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b95d52-e243-414a-baa7-334c47693325 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:24.477160 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b95d52-e243-414a-baa7-334c47693325 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:24.489356 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34317792-e1a0-4416-8640-abe480f99b06 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:24.495548 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b95d52-e243-414a-baa7-334c47693325 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 11:06:24.503076 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-d9b95d52-e243-414a-baa7-334c47693325 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 29 11:06:24.503076 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:06:24.503076 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:06:24.503076 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:06:24.503076 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:06:24.503076 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:06:24.503076 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:06:24.503076 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:06:24.503076 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:06:24.503076 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:06:24.503076 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:06:24.503076 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:06:24.503076 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 11:06:24.503076 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 11:06:24.503076 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 11:06:24.503076 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 11:06:24.504346 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 11:06:24.504346 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 11:06:24.504346 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 11:06:24.504346 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 11:06:24.504346 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:06:24.504346 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:06:24.504346 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:06:24.504346 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:06:24.504346 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:06:24.504346 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:06:24.504346 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:06:24.504346 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jun 29 11:06:24.504346 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:06:24.504346 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:06:24.504346 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:06:24.504346 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:06:24.505160 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:06:24.505160 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:06:24.505160 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:06:24.505160 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:06:24.505160 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 11:06:24.505160 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 11:06:24.505160 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 11:06:24.505160 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 11:06:24.505160 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 11:06:24.505160 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 11:06:24.505160 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 29 11:06:24.505160 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jun 29 11:06:24.505160 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 188/361] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Jun 29 11:06:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:06:24.506632 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e24797-5226-46e3-abfd-692db335f893 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:24.506775 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e24797-5226-46e3-abfd-692db335f893 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:24.507049 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e24797-5226-46e3-abfd-692db335f893 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:24.528646 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-a6e24797-5226-46e3-abfd-692db335f893 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:06:24.529851 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-a6e24797-5226-46e3-abfd-692db335f893 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:06:24.532784 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd32ddf-0cee-4aea-9b97-0c422333616c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:24.533644 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd32ddf-0cee-4aea-9b97-0c422333616c None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:24.533929 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd32ddf-0cee-4aea-9b97-0c422333616c None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:24.534207 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd32ddf-0cee-4aea-9b97-0c422333616c None admin] PATH_INFO: `/v3/groups/8ec7aa0c6a3d47889a0b287a0fc483ad` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:24.536178 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd32ddf-0cee-4aea-9b97-0c422333616c None admin] RBAC: Authorizing `identity:get_group(group_id=8ec7aa0c6a3d47889a0b287a0fc483ad)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:24.537691 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd32ddf-0cee-4aea-9b97-0c422333616c None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:24.539855 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 174/362] 10.222.0.44 () {56 vars in 1180 bytes} [Wed Jun 29 11:06:24 2022] GET /identity/v3/groups/8ec7aa0c6a3d47889a0b287a0fc483ad => generated 226 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:24.540125 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd32ddf-0cee-4aea-9b97-0c422333616c None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:24.615785 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 189/363] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:06:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 11:06:24.616277 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e24797-5226-46e3-abfd-692db335f893 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:24.666851 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd32ddf-0cee-4aea-9b97-0c422333616c None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:24.679501 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:24.714464 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9de4165-402c-4fe6-820c-9adac80f642c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:24.715141 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9de4165-402c-4fe6-820c-9adac80f642c None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:24.715744 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9de4165-402c-4fe6-820c-9adac80f642c None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:24.716066 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9de4165-402c-4fe6-820c-9adac80f642c None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:24.717122 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9de4165-402c-4fe6-820c-9adac80f642c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:06:24.717434 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9de4165-402c-4fe6-820c-9adac80f642c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:24.719394 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9de4165-402c-4fe6-820c-9adac80f642c None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:24.730591 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 175/364] 10.222.0.44 () {56 vars in 1111 bytes} [Wed Jun 29 11:06:24 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:24.731484 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9de4165-402c-4fe6-820c-9adac80f642c None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:24.736856 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:24.744569 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e24797-5226-46e3-abfd-692db335f893 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:24.803417 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc54e69-9ced-4e52-a591-c1440390a40b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:24.804533 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc54e69-9ced-4e52-a591-c1440390a40b None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:24.805226 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc54e69-9ced-4e52-a591-c1440390a40b None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:24.805513 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc54e69-9ced-4e52-a591-c1440390a40b None admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:24.806282 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc54e69-9ced-4e52-a591-c1440390a40b None admin] RBAC: Adding query filter params (name=member) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:06:24.806845 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc54e69-9ced-4e52-a591-c1440390a40b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:24.809067 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc54e69-9ced-4e52-a591-c1440390a40b None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:24.823167 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 190/365] 10.222.0.44 () {56 vars in 1101 bytes} [Wed Jun 29 11:06:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:24.824447 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc54e69-9ced-4e52-a591-c1440390a40b None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:24.861348 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9de4165-402c-4fe6-820c-9adac80f642c None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:24.901196 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c9188c-e342-4e96-b783-95bae8d34260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:24.901953 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c9188c-e342-4e96-b783-95bae8d34260 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:24.902308 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c9188c-e342-4e96-b783-95bae8d34260 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:24.902659 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c9188c-e342-4e96-b783-95bae8d34260 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:24.903572 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c9188c-e342-4e96-b783-95bae8d34260 None admin] RBAC: Adding query filter params (group.id=8ec7aa0c6a3d47889a0b287a0fc483ad, role.id=fc91ce4ad4a44cc3b05d0936754e87ec, scope.project.id=3b98d22914de4e3b8bca7fe0a0b51357) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:06:24.903986 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c9188c-e342-4e96-b783-95bae8d34260 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:24.905495 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c9188c-e342-4e96-b783-95bae8d34260 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:24.911997 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 176/366] 10.222.0.44 () {56 vars in 1376 bytes} [Wed Jun 29 11:06:24 2022] GET /identity/v3/role_assignments?group.id=8ec7aa0c6a3d47889a0b287a0fc483ad&scope.project.id=3b98d22914de4e3b8bca7fe0a0b51357&role.id=fc91ce4ad4a44cc3b05d0936754e87ec => generated 260 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:24.912718 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c9188c-e342-4e96-b783-95bae8d34260 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:24.956125 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc54e69-9ced-4e52-a591-c1440390a40b None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:24.962915 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:24.998520 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe4a1d8-2470-4716-a026-226fc71d7194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:24.999821 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe4a1d8-2470-4716-a026-226fc71d7194 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:25.000142 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe4a1d8-2470-4716-a026-226fc71d7194 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:25.000418 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe4a1d8-2470-4716-a026-226fc71d7194 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:25.013921 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe4a1d8-2470-4716-a026-226fc71d7194 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 11:06:25.022257 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-9fe4a1d8-2470-4716-a026-226fc71d7194 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 11:06:25.022257 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:06:25.022257 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:06:25.022257 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:06:25.022257 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:06:25.022257 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:06:25.022257 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:06:25.022257 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:06:25.022257 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:06:25.022257 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:06:25.022257 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:06:25.022257 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:06:25.022257 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 11:06:25.022257 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 11:06:25.022257 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 11:06:25.022257 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 11:06:25.023485 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 11:06:25.023485 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 11:06:25.023485 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 11:06:25.023485 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 11:06:25.023485 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:06:25.023485 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:06:25.023485 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:06:25.023485 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:06:25.023485 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:06:25.023485 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:06:25.023485 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:06:25.023485 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jun 29 11:06:25.023485 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:06:25.023485 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:06:25.023485 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:06:25.023485 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:06:25.024556 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:06:25.024556 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:06:25.024556 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:06:25.024556 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:06:25.024556 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 11:06:25.024556 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 11:06:25.024556 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 11:06:25.024556 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 11:06:25.024556 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 11:06:25.024556 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 11:06:25.024556 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 11:06:25.024556 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jun 29 11:06:25.026158 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 191/367] 10.222.0.44 () {56 vars in 1096 bytes} [Wed Jun 29 11:06:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:06:25.026561 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe4a1d8-2470-4716-a026-226fc71d7194 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:25.057044 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:25.071343 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c9188c-e342-4e96-b783-95bae8d34260 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:25.120487 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70392cec-2f48-455d-8209-4062e1488cd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:25.121501 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70392cec-2f48-455d-8209-4062e1488cd2 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:25.121986 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70392cec-2f48-455d-8209-4062e1488cd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:25.122486 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70392cec-2f48-455d-8209-4062e1488cd2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:25.171079 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe4a1d8-2470-4716-a026-226fc71d7194 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:25.211156 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70392cec-2f48-455d-8209-4062e1488cd2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:25.212660 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70392cec-2f48-455d-8209-4062e1488cd2 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:25.213480 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1c4d16-e608-45ec-9514-bbe99f850d5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:25.214609 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1c4d16-e608-45ec-9514-bbe99f850d5d None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:25.215091 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1c4d16-e608-45ec-9514-bbe99f850d5d None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:25.216277 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1c4d16-e608-45ec-9514-bbe99f850d5d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:25.249413 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:25.304581 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1c4d16-e608-45ec-9514-bbe99f850d5d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:25.306479 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1c4d16-e608-45ec-9514-bbe99f850d5d None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:25.324662 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 177/368] 10.222.0.44 () {58 vars in 1282 bytes} [Wed Jun 29 11:06:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 29 11:06:25.325898 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70392cec-2f48-455d-8209-4062e1488cd2 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:25.331792 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:25.448730 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70392cec-2f48-455d-8209-4062e1488cd2 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:25.449196 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 192/369] 10.222.0.44 () {58 vars in 1282 bytes} [Wed Jun 29 11:06:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 29 11:06:25.450040 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1c4d16-e608-45ec-9514-bbe99f850d5d None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:25.491352 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c0178f-0d87-4efd-bf63-89b8056cd919 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:25.492413 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c0178f-0d87-4efd-bf63-89b8056cd919 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:25.492906 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c0178f-0d87-4efd-bf63-89b8056cd919 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:25.493396 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c0178f-0d87-4efd-bf63-89b8056cd919 None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:25.494328 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c0178f-0d87-4efd-bf63-89b8056cd919 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:06:25.494836 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c0178f-0d87-4efd-bf63-89b8056cd919 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:25.496434 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c0178f-0d87-4efd-bf63-89b8056cd919 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:25.499329 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:25.507378 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 178/370] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Jun 29 11:06:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:25.509636 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c0178f-0d87-4efd-bf63-89b8056cd919 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:25.583758 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1c4d16-e608-45ec-9514-bbe99f850d5d None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:25.605956 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:25.629763 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c0178f-0d87-4efd-bf63-89b8056cd919 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:25.629978 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa3f636-6cae-4f5a-8827-42198d8cf934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:25.631129 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa3f636-6cae-4f5a-8827-42198d8cf934 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:25.631568 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa3f636-6cae-4f5a-8827-42198d8cf934 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:25.632363 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa3f636-6cae-4f5a-8827-42198d8cf934 None admin] PATH_INFO: `/v3/projects/d36c6c9eb3234e82bb6a8537851eb336` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:25.634684 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa3f636-6cae-4f5a-8827-42198d8cf934 None admin] RBAC: Authorizing `identity:get_project(project_id=d36c6c9eb3234e82bb6a8537851eb336)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:25.636307 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa3f636-6cae-4f5a-8827-42198d8cf934 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:25.639025 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 193/371] 10.222.0.44 () {56 vars in 1186 bytes} [Wed Jun 29 11:06:25 2022] GET /identity/v3/projects/d36c6c9eb3234e82bb6a8537851eb336 => generated 302 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:25.640081 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa3f636-6cae-4f5a-8827-42198d8cf934 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:25.680744 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a0c3d7-4ede-49cf-8fd7-ffc05c5f3534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:25.681639 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a0c3d7-4ede-49cf-8fd7-ffc05c5f3534 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:25.681920 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a0c3d7-4ede-49cf-8fd7-ffc05c5f3534 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:25.682192 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a0c3d7-4ede-49cf-8fd7-ffc05c5f3534 None admin] PATH_INFO: `/v3/projects/d36c6c9eb3234e82bb6a8537851eb336` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:25.683512 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a0c3d7-4ede-49cf-8fd7-ffc05c5f3534 None admin] RBAC: Authorizing `identity:get_project(project_id=d36c6c9eb3234e82bb6a8537851eb336)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:25.685233 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a0c3d7-4ede-49cf-8fd7-ffc05c5f3534 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:25.687469 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 179/372] 10.222.0.44 () {56 vars in 1186 bytes} [Wed Jun 29 11:06:25 2022] GET /identity/v3/projects/d36c6c9eb3234e82bb6a8537851eb336 => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:25.688534 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a0c3d7-4ede-49cf-8fd7-ffc05c5f3534 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:25.769959 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:25.783890 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa3f636-6cae-4f5a-8827-42198d8cf934 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:25.817139 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a0c3d7-4ede-49cf-8fd7-ffc05c5f3534 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:25.826022 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5da676-a6a8-4403-a594-ae7dde5dd736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:25.827314 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5da676-a6a8-4403-a594-ae7dde5dd736 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:25.827824 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5da676-a6a8-4403-a594-ae7dde5dd736 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:25.828344 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5da676-a6a8-4403-a594-ae7dde5dd736 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:25.862101 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ddefc2-e91d-40e4-879b-cba41334c232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:25.863010 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ddefc2-e91d-40e4-879b-cba41334c232 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:25.863308 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ddefc2-e91d-40e4-879b-cba41334c232 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:25.863816 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ddefc2-e91d-40e4-879b-cba41334c232 None admin] PATH_INFO: `/v3/groups/8ec7aa0c6a3d47889a0b287a0fc483ad` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:25.865470 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ddefc2-e91d-40e4-879b-cba41334c232 None admin] RBAC: Authorizing `identity:get_group(group_id=8ec7aa0c6a3d47889a0b287a0fc483ad)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:25.867184 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ddefc2-e91d-40e4-879b-cba41334c232 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:25.869290 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 180/373] 10.222.0.44 () {56 vars in 1180 bytes} [Wed Jun 29 11:06:25 2022] GET /identity/v3/groups/8ec7aa0c6a3d47889a0b287a0fc483ad => generated 226 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:25.870753 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ddefc2-e91d-40e4-879b-cba41334c232 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:25.890107 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:25.909577 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5da676-a6a8-4403-a594-ae7dde5dd736 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:25.911975 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5da676-a6a8-4403-a594-ae7dde5dd736 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:26.009077 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ddefc2-e91d-40e4-879b-cba41334c232 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:26.034781 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 194/374] 10.222.0.44 () {58 vars in 1282 bytes} [Wed Jun 29 11:06:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 29 11:06:26.036458 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5da676-a6a8-4403-a594-ae7dde5dd736 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:26.060842 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c2e9bc-e5d0-4e9c-af07-2458db880721 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:26.063870 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c2e9bc-e5d0-4e9c-af07-2458db880721 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:26.063870 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c2e9bc-e5d0-4e9c-af07-2458db880721 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:26.063870 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c2e9bc-e5d0-4e9c-af07-2458db880721 None admin] PATH_INFO: `/v3/groups/8ec7aa0c6a3d47889a0b287a0fc483ad` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:26.063870 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c2e9bc-e5d0-4e9c-af07-2458db880721 None admin] RBAC: Authorizing `identity:get_group(group_id=8ec7aa0c6a3d47889a0b287a0fc483ad)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:26.064987 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c2e9bc-e5d0-4e9c-af07-2458db880721 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:26.067044 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 181/375] 10.222.0.44 () {56 vars in 1180 bytes} [Wed Jun 29 11:06:25 2022] GET /identity/v3/groups/8ec7aa0c6a3d47889a0b287a0fc483ad => generated 226 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:26.067819 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c2e9bc-e5d0-4e9c-af07-2458db880721 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:26.072097 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:26.159094 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:26.181914 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5da676-a6a8-4403-a594-ae7dde5dd736 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:26.202396 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c2e9bc-e5d0-4e9c-af07-2458db880721 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:26.234712 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db46700a-b547-4b53-8ec8-b52c29d72a11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:26.235877 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db46700a-b547-4b53-8ec8-b52c29d72a11 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:26.236361 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db46700a-b547-4b53-8ec8-b52c29d72a11 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:26.236787 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db46700a-b547-4b53-8ec8-b52c29d72a11 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:26.237625 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db46700a-b547-4b53-8ec8-b52c29d72a11 None admin] RBAC: Adding query filter params (group.id=8ec7aa0c6a3d47889a0b287a0fc483ad, role.id=fc91ce4ad4a44cc3b05d0936754e87ec, scope.project.id=d36c6c9eb3234e82bb6a8537851eb336) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:06:26.237935 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db46700a-b547-4b53-8ec8-b52c29d72a11 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:26.239364 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db46700a-b547-4b53-8ec8-b52c29d72a11 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:26.248765 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 195/376] 10.222.0.44 () {56 vars in 1376 bytes} [Wed Jun 29 11:06:26 2022] GET /identity/v3/role_assignments?group.id=8ec7aa0c6a3d47889a0b287a0fc483ad&scope.project.id=d36c6c9eb3234e82bb6a8537851eb336&role.id=fc91ce4ad4a44cc3b05d0936754e87ec => generated 260 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:26.248974 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db46700a-b547-4b53-8ec8-b52c29d72a11 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:26.266699 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73b95f7-73f6-44fd-bb36-90cee53afc5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:26.271239 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73b95f7-73f6-44fd-bb36-90cee53afc5e None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:26.273216 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73b95f7-73f6-44fd-bb36-90cee53afc5e None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:26.273807 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73b95f7-73f6-44fd-bb36-90cee53afc5e None admin] PATH_INFO: `/v3/projects/37897f6664b94d219c11b87aa269041a` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:26.275768 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73b95f7-73f6-44fd-bb36-90cee53afc5e None admin] RBAC: Authorizing `identity:get_project(project_id=37897f6664b94d219c11b87aa269041a)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:26.278351 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73b95f7-73f6-44fd-bb36-90cee53afc5e None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:26.280921 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 182/377] 10.222.0.44 () {56 vars in 1186 bytes} [Wed Jun 29 11:06:26 2022] GET /identity/v3/projects/37897f6664b94d219c11b87aa269041a => generated 344 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:26.283697 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f1b004-bcea-4c8b-920e-817002355b75 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:26.284340 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f1b004-bcea-4c8b-920e-817002355b75 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:26.284877 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f1b004-bcea-4c8b-920e-817002355b75 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:26.293051 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 183/378] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:06:26 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:06:26.294328 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f1b004-bcea-4c8b-920e-817002355b75 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:26.417470 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:26.437025 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db46700a-b547-4b53-8ec8-b52c29d72a11 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:26.456764 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f1b004-bcea-4c8b-920e-817002355b75 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:26.482253 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421b5217-fc32-4549-9040-9312f88c75df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:26.482915 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421b5217-fc32-4549-9040-9312f88c75df None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:26.483198 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421b5217-fc32-4549-9040-9312f88c75df None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:26.483631 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421b5217-fc32-4549-9040-9312f88c75df None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:26.485468 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421b5217-fc32-4549-9040-9312f88c75df None admin] RBAC: Adding query filter params (group.id=8ec7aa0c6a3d47889a0b287a0fc483ad, role.id=e1c88eebe911432cab6a07fbeb84653b, scope.project.id=d36c6c9eb3234e82bb6a8537851eb336) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:06:26.485796 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421b5217-fc32-4549-9040-9312f88c75df None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:26.487553 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421b5217-fc32-4549-9040-9312f88c75df None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:26.494445 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 196/379] 10.222.0.44 () {56 vars in 1376 bytes} [Wed Jun 29 11:06:26 2022] GET /identity/v3/role_assignments?group.id=8ec7aa0c6a3d47889a0b287a0fc483ad&scope.project.id=d36c6c9eb3234e82bb6a8537851eb336&role.id=e1c88eebe911432cab6a07fbeb84653b => generated 260 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:26.495694 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3784b5f6-1cf2-41df-a5cc-013f95e50ac7 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:26.497860 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b05a89-e828-4271-b31a-00e4bcfb3d57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:26.499002 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b05a89-e828-4271-b31a-00e4bcfb3d57 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:26.500656 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b05a89-e828-4271-b31a-00e4bcfb3d57 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:26.501214 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b05a89-e828-4271-b31a-00e4bcfb3d57 None admin] PATH_INFO: `/v3/groups/9451dcb3c55545be80b6512695c19e9b` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:26.504170 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3784b5f6-1cf2-41df-a5cc-013f95e50ac7 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:26.504528 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3784b5f6-1cf2-41df-a5cc-013f95e50ac7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:26.508981 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b05a89-e828-4271-b31a-00e4bcfb3d57 None admin] RBAC: Authorizing `identity:get_group(group_id=9451dcb3c55545be80b6512695c19e9b)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:26.510846 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b05a89-e828-4271-b31a-00e4bcfb3d57 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:26.513147 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 184/380] 10.222.0.44 () {56 vars in 1180 bytes} [Wed Jun 29 11:06:26 2022] GET /identity/v3/groups/9451dcb3c55545be80b6512695c19e9b => generated 229 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:26.519568 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b05a89-e828-4271-b31a-00e4bcfb3d57 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:26.545105 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-3784b5f6-1cf2-41df-a5cc-013f95e50ac7 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:06:26.546304 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-3784b5f6-1cf2-41df-a5cc-013f95e50ac7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:06:26.559013 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:26.734777 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 197/381] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:06:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 11:06:26.764771 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3784b5f6-1cf2-41df-a5cc-013f95e50ac7 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:26.770644 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b05a89-e828-4271-b31a-00e4bcfb3d57 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:26.877252 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74888a8c-4804-4770-87d3-e6b2229ba24b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:26.877863 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74888a8c-4804-4770-87d3-e6b2229ba24b None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:26.878129 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74888a8c-4804-4770-87d3-e6b2229ba24b None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:26.878409 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74888a8c-4804-4770-87d3-e6b2229ba24b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:26.879258 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74888a8c-4804-4770-87d3-e6b2229ba24b None admin] RBAC: Adding query filter params (group.id=9451dcb3c55545be80b6512695c19e9b, role.id=e95c2ce63e114cc19a9dda6ad21e794f, scope.project.id=37897f6664b94d219c11b87aa269041a) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:06:26.879572 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74888a8c-4804-4770-87d3-e6b2229ba24b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:26.881095 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74888a8c-4804-4770-87d3-e6b2229ba24b None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:26.883334 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:26.901177 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 185/382] 10.222.0.44 () {56 vars in 1376 bytes} [Wed Jun 29 11:06:26 2022] GET /identity/v3/role_assignments?group.id=9451dcb3c55545be80b6512695c19e9b&scope.project.id=37897f6664b94d219c11b87aa269041a&role.id=e95c2ce63e114cc19a9dda6ad21e794f => generated 260 bytes in 382 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 11:06:26.962835 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3784b5f6-1cf2-41df-a5cc-013f95e50ac7 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:27.004663 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb9f359-b991-4e51-957b-8bb02841ccfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:27.005499 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb9f359-b991-4e51-957b-8bb02841ccfd None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:27.005821 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb9f359-b991-4e51-957b-8bb02841ccfd None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:27.006104 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb9f359-b991-4e51-957b-8bb02841ccfd None admin] PATH_INFO: `/v3/roles/member` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:27.020256 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb9f359-b991-4e51-957b-8bb02841ccfd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 11:06:27.028357 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-fcb9f359-b991-4e51-957b-8bb02841ccfd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 29 11:06:27.028357 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:06:27.028357 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:06:27.028357 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:06:27.028357 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:06:27.028357 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:06:27.028357 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:06:27.028357 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:06:27.028357 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:06:27.028357 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:06:27.028357 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:06:27.028357 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:06:27.028357 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 11:06:27.028357 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 11:06:27.028357 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 11:06:27.028357 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 11:06:27.029278 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 11:06:27.029278 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 11:06:27.029278 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 11:06:27.029278 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 11:06:27.029278 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:06:27.029278 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:06:27.029278 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:06:27.029278 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:06:27.029278 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:06:27.029278 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:06:27.029278 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:06:27.029278 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jun 29 11:06:27.029278 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:06:27.029278 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:06:27.029278 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:06:27.029278 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:06:27.030511 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:06:27.030511 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:06:27.030511 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:06:27.030511 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:06:27.030511 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 11:06:27.030511 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 11:06:27.030511 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 11:06:27.030511 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 11:06:27.030511 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 11:06:27.030511 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 11:06:27.030511 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 29 11:06:27.030511 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jun 29 11:06:27.030511 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 198/383] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Jun 29 11:06:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:06:27.038624 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74888a8c-4804-4770-87d3-e6b2229ba24b None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:27.184673 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74888a8c-4804-4770-87d3-e6b2229ba24b None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:27.229614 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95e3e47-f3b3-4a60-a127-57c31abc4150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:27.230292 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95e3e47-f3b3-4a60-a127-57c31abc4150 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:27.230576 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95e3e47-f3b3-4a60-a127-57c31abc4150 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:27.230855 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95e3e47-f3b3-4a60-a127-57c31abc4150 None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:27.231605 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95e3e47-f3b3-4a60-a127-57c31abc4150 None admin] RBAC: Adding query filter params (name=member) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:06:27.231946 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95e3e47-f3b3-4a60-a127-57c31abc4150 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:27.233310 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95e3e47-f3b3-4a60-a127-57c31abc4150 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:27.249551 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 186/384] 10.222.0.44 () {56 vars in 1101 bytes} [Wed Jun 29 11:06:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:27.255495 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb9f359-b991-4e51-957b-8bb02841ccfd None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:27.260863 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:27.389986 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb9f359-b991-4e51-957b-8bb02841ccfd None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:27.431475 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c2e7e8-554c-4b5e-a916-9e571c9f18b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:27.432725 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c2e7e8-554c-4b5e-a916-9e571c9f18b2 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:27.433213 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c2e7e8-554c-4b5e-a916-9e571c9f18b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:27.433719 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c2e7e8-554c-4b5e-a916-9e571c9f18b2 None admin] PATH_INFO: `/v3/groups/8ec7aa0c6a3d47889a0b287a0fc483ad` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:27.435196 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c2e7e8-554c-4b5e-a916-9e571c9f18b2 None admin] RBAC: Authorizing `identity:get_group(group_id=8ec7aa0c6a3d47889a0b287a0fc483ad)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:27.436892 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c2e7e8-554c-4b5e-a916-9e571c9f18b2 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:27.438730 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 199/385] 10.222.0.44 () {56 vars in 1180 bytes} [Wed Jun 29 11:06:27 2022] GET /identity/v3/groups/8ec7aa0c6a3d47889a0b287a0fc483ad => generated 226 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:27.444976 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95e3e47-f3b3-4a60-a127-57c31abc4150 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:27.540039 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:27.542462 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3be083-fe94-4d7f-aa29-5a00aeb9f509 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:27.542966 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3be083-fe94-4d7f-aa29-5a00aeb9f509 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:27.543447 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3be083-fe94-4d7f-aa29-5a00aeb9f509 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:27.545268 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 200/386] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:06:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 29 11:06:27.554335 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8599d4ac-e923-4d1d-ab0c-5c7c5244f97c None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:27.554622 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8599d4ac-e923-4d1d-ab0c-5c7c5244f97c None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:27.554903 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8599d4ac-e923-4d1d-ab0c-5c7c5244f97c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:27.582886 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-8599d4ac-e923-4d1d-ab0c-5c7c5244f97c None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:06:27.584139 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-8599d4ac-e923-4d1d-ab0c-5c7c5244f97c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:06:27.585209 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95e3e47-f3b3-4a60-a127-57c31abc4150 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:27.627628 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f561603e-2b85-4e67-804b-5afd7cb892a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:27.628535 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f561603e-2b85-4e67-804b-5afd7cb892a3 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:27.628818 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f561603e-2b85-4e67-804b-5afd7cb892a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:27.629096 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f561603e-2b85-4e67-804b-5afd7cb892a3 None admin] PATH_INFO: `/v3/projects/3b98d22914de4e3b8bca7fe0a0b51357` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:27.631141 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f561603e-2b85-4e67-804b-5afd7cb892a3 None admin] RBAC: Authorizing `identity:get_project(project_id=3b98d22914de4e3b8bca7fe0a0b51357)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:27.632613 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f561603e-2b85-4e67-804b-5afd7cb892a3 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:27.635530 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 187/387] 10.222.0.44 () {56 vars in 1186 bytes} [Wed Jun 29 11:06:27 2022] GET /identity/v3/projects/3b98d22914de4e3b8bca7fe0a0b51357 => generated 298 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:27.643532 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f561603e-2b85-4e67-804b-5afd7cb892a3 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:27.688265 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:27.694430 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 201/388] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:06:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 11:06:27.734620 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8599d4ac-e923-4d1d-ab0c-5c7c5244f97c None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:27.809469 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f561603e-2b85-4e67-804b-5afd7cb892a3 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:27.856300 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:27.858696 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8599d4ac-e923-4d1d-ab0c-5c7c5244f97c None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:27.877109 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a162fdc-4f1e-4f2d-ad29-e03e00f93094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:27.879692 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a162fdc-4f1e-4f2d-ad29-e03e00f93094 None admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:27.879692 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a162fdc-4f1e-4f2d-ad29-e03e00f93094 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:27.879692 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a162fdc-4f1e-4f2d-ad29-e03e00f93094 None admin] PATH_INFO: `/v3/projects/3b98d22914de4e3b8bca7fe0a0b51357/groups/8ec7aa0c6a3d47889a0b287a0fc483ad/roles/e1c88eebe911432cab6a07fbeb84653b` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:27.885156 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a162fdc-4f1e-4f2d-ad29-e03e00f93094 None admin] RBAC: Authorizing `identity:create_grant(project_id=3b98d22914de4e3b8bca7fe0a0b51357, group_id=8ec7aa0c6a3d47889a0b287a0fc483ad, role_id=e1c88eebe911432cab6a07fbeb84653b)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:27.886598 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a162fdc-4f1e-4f2d-ad29-e03e00f93094 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:27.894137 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafc1ce6-b3ec-45cd-b494-7c596c81e54f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:27.894952 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafc1ce6-b3ec-45cd-b494-7c596c81e54f None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:27.895270 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafc1ce6-b3ec-45cd-b494-7c596c81e54f None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:27.895558 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafc1ce6-b3ec-45cd-b494-7c596c81e54f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:27.906412 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafc1ce6-b3ec-45cd-b494-7c596c81e54f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 11:06:27.925929 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-cafc1ce6-b3ec-45cd-b494-7c596c81e54f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 29 11:06:27.925929 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:06:27.925929 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:06:27.925929 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:06:27.925929 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:06:27.925929 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:06:27.925929 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:06:27.925929 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:06:27.925929 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:06:27.925929 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:06:27.925929 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:06:27.925929 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:06:27.925929 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 11:06:27.925929 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 11:06:27.925929 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 11:06:27.925929 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 11:06:27.926805 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 11:06:27.926805 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 11:06:27.926805 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 11:06:27.926805 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 11:06:27.926805 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:06:27.926805 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:06:27.926805 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:06:27.926805 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:06:27.926805 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:06:27.926805 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:06:27.926805 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:06:27.926805 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jun 29 11:06:27.926805 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:06:27.926805 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:06:27.926805 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:06:27.926805 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:06:27.928282 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:06:27.928282 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:06:27.928282 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:06:27.928282 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:06:27.928282 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 11:06:27.928282 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 11:06:27.928282 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 11:06:27.928282 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 11:06:27.928282 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 11:06:27.928282 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 11:06:27.928282 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 29 11:06:27.928282 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jun 29 11:06:27.928282 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 202/389] 10.222.0.44 () {56 vars in 1114 bytes} [Wed Jun 29 11:06:27 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 11:06:27.934476 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafc1ce6-b3ec-45cd-b494-7c596c81e54f None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:27.982433 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 188/390] 10.222.0.44 () {58 vars in 1442 bytes} [Wed Jun 29 11:06:27 2022] PUT /identity/v3/projects/3b98d22914de4e3b8bca7fe0a0b51357/groups/8ec7aa0c6a3d47889a0b287a0fc483ad/roles/e1c88eebe911432cab6a07fbeb84653b => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:06:28.004454 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:28.055737 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafc1ce6-b3ec-45cd-b494-7c596c81e54f None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:28.102791 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179080d1-5f53-4c6b-b01a-738bfe6d6656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:28.103477 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179080d1-5f53-4c6b-b01a-738bfe6d6656 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:28.103759 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179080d1-5f53-4c6b-b01a-738bfe6d6656 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:28.104056 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179080d1-5f53-4c6b-b01a-738bfe6d6656 None admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:28.104886 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179080d1-5f53-4c6b-b01a-738bfe6d6656 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:06:28.105195 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179080d1-5f53-4c6b-b01a-738bfe6d6656 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:28.106568 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179080d1-5f53-4c6b-b01a-738bfe6d6656 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:28.115906 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 203/391] 10.222.0.44 () {56 vars in 1111 bytes} [Wed Jun 29 11:06:27 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:28.121538 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a162fdc-4f1e-4f2d-ad29-e03e00f93094 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:28.271737 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a162fdc-4f1e-4f2d-ad29-e03e00f93094 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:28.314001 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4854ac-2b40-4408-ab96-09ff69474c83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:28.314865 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4854ac-2b40-4408-ab96-09ff69474c83 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:28.315150 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4854ac-2b40-4408-ab96-09ff69474c83 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:28.315428 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4854ac-2b40-4408-ab96-09ff69474c83 None admin] PATH_INFO: `/v3/groups/8ec7aa0c6a3d47889a0b287a0fc483ad` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:28.316664 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4854ac-2b40-4408-ab96-09ff69474c83 None admin] RBAC: Authorizing `identity:get_group(group_id=8ec7aa0c6a3d47889a0b287a0fc483ad)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:28.318646 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4854ac-2b40-4408-ab96-09ff69474c83 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:28.323936 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 189/392] 10.222.0.44 () {56 vars in 1180 bytes} [Wed Jun 29 11:06:28 2022] GET /identity/v3/groups/8ec7aa0c6a3d47889a0b287a0fc483ad => generated 226 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:28.326451 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179080d1-5f53-4c6b-b01a-738bfe6d6656 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:28.431814 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179080d1-5f53-4c6b-b01a-738bfe6d6656 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:28.454670 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:28.466496 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6028ef-d6e2-4490-88f7-0c252992fb8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:28.467353 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6028ef-d6e2-4490-88f7-0c252992fb8e None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:28.467640 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6028ef-d6e2-4490-88f7-0c252992fb8e None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:28.467989 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6028ef-d6e2-4490-88f7-0c252992fb8e None admin] PATH_INFO: `/v3/projects/3b98d22914de4e3b8bca7fe0a0b51357` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:28.469202 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6028ef-d6e2-4490-88f7-0c252992fb8e None admin] RBAC: Authorizing `identity:get_project(project_id=3b98d22914de4e3b8bca7fe0a0b51357)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:28.470631 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6028ef-d6e2-4490-88f7-0c252992fb8e None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:28.472595 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 204/393] 10.222.0.44 () {56 vars in 1186 bytes} [Wed Jun 29 11:06:28 2022] GET /identity/v3/projects/3b98d22914de4e3b8bca7fe0a0b51357 => generated 298 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:28.478483 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4854ac-2b40-4408-ab96-09ff69474c83 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:28.543264 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:28.593630 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4854ac-2b40-4408-ab96-09ff69474c83 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:28.639471 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31927a61-59af-4888-8dc9-d9e575e13a3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:28.640389 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31927a61-59af-4888-8dc9-d9e575e13a3f None admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:28.640684 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31927a61-59af-4888-8dc9-d9e575e13a3f None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:28.641041 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31927a61-59af-4888-8dc9-d9e575e13a3f None admin] PATH_INFO: `/v3/projects/3b98d22914de4e3b8bca7fe0a0b51357/groups/8ec7aa0c6a3d47889a0b287a0fc483ad/roles/fc91ce4ad4a44cc3b05d0936754e87ec` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:28.648116 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31927a61-59af-4888-8dc9-d9e575e13a3f None admin] RBAC: Authorizing `identity:create_grant(project_id=3b98d22914de4e3b8bca7fe0a0b51357, group_id=8ec7aa0c6a3d47889a0b287a0fc483ad, role_id=fc91ce4ad4a44cc3b05d0936754e87ec)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:28.649576 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31927a61-59af-4888-8dc9-d9e575e13a3f None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:28.725181 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 190/394] 10.222.0.44 () {58 vars in 1442 bytes} [Wed Jun 29 11:06:28 2022] PUT /identity/v3/projects/3b98d22914de4e3b8bca7fe0a0b51357/groups/8ec7aa0c6a3d47889a0b287a0fc483ad/roles/fc91ce4ad4a44cc3b05d0936754e87ec => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:06:29.069465 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6158a8-f727-46f2-bacd-de09d38aa552 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:29.069990 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6158a8-f727-46f2-bacd-de09d38aa552 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:29.069990 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6158a8-f727-46f2-bacd-de09d38aa552 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:29.071653 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 205/395] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:06:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:06:29.077468 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a08070-b435-48fc-8f03-f7af071636c2 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:29.077872 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a08070-b435-48fc-8f03-f7af071636c2 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:29.078034 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a08070-b435-48fc-8f03-f7af071636c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:29.095432 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-50a08070-b435-48fc-8f03-f7af071636c2 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:06:29.096681 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-50a08070-b435-48fc-8f03-f7af071636c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:06:29.176011 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 191/396] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:06:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 29 11:06:29.191954 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6158a8-f727-46f2-bacd-de09d38aa552 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:29.283396 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3338015d-1b73-4f36-97b7-61d83e49c621 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:29.283714 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3338015d-1b73-4f36-97b7-61d83e49c621 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:29.283997 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3338015d-1b73-4f36-97b7-61d83e49c621 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:29.285645 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 192/397] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:06:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:06:29.290633 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157e949f-3b9f-46f3-a5d1-31387779cc5e None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:29.291075 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157e949f-3b9f-46f3-a5d1-31387779cc5e None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:29.291495 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157e949f-3b9f-46f3-a5d1-31387779cc5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:29.314213 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-157e949f-3b9f-46f3-a5d1-31387779cc5e None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:06:29.316407 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-157e949f-3b9f-46f3-a5d1-31387779cc5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:06:29.356349 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6158a8-f727-46f2-bacd-de09d38aa552 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:29.406425 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:29.422576 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95c52d7-47ab-48d6-814d-7283fbb5c595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:29.423620 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95c52d7-47ab-48d6-814d-7283fbb5c595 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:29.424077 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95c52d7-47ab-48d6-814d-7283fbb5c595 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:29.424551 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95c52d7-47ab-48d6-814d-7283fbb5c595 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:29.448785 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95c52d7-47ab-48d6-814d-7283fbb5c595 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 11:06:29.454554 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 193/398] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:06:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 29 11:06:29.460079 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-f95c52d7-47ab-48d6-814d-7283fbb5c595 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 29 11:06:29.460079 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:06:29.460079 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:06:29.460079 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:06:29.460079 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:06:29.460079 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:06:29.460079 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:06:29.460079 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:06:29.460079 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:06:29.460079 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:06:29.460079 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:06:29.460079 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:06:29.460079 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 11:06:29.460079 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 11:06:29.460079 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 11:06:29.460079 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 11:06:29.461297 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 11:06:29.461297 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 11:06:29.461297 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 11:06:29.461297 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 11:06:29.461297 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:06:29.461297 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:06:29.461297 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:06:29.461297 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:06:29.461297 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:06:29.461297 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:06:29.461297 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:06:29.461297 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jun 29 11:06:29.461297 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:06:29.461297 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:06:29.461297 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:06:29.461297 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:06:29.462394 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:06:29.462394 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:06:29.462394 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:06:29.462394 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:06:29.462394 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 11:06:29.462394 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 11:06:29.462394 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 11:06:29.462394 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 11:06:29.462394 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 11:06:29.462394 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 11:06:29.462394 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 29 11:06:29.462394 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jun 29 11:06:29.469361 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 206/399] 10.222.0.44 () {56 vars in 1114 bytes} [Wed Jun 29 11:06:29 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:06:29.470817 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157e949f-3b9f-46f3-a5d1-31387779cc5e None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:29.494489 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95c52d7-47ab-48d6-814d-7283fbb5c595 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:29.559750 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:29.601483 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157e949f-3b9f-46f3-a5d1-31387779cc5e None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:29.625095 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95c52d7-47ab-48d6-814d-7283fbb5c595 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:29.639338 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5316bb66-1e2e-4cc5-8fa2-ff2d38071a9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:29.640269 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5316bb66-1e2e-4cc5-8fa2-ff2d38071a9c None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:29.640541 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5316bb66-1e2e-4cc5-8fa2-ff2d38071a9c None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:29.640935 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5316bb66-1e2e-4cc5-8fa2-ff2d38071a9c None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:29.641785 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5316bb66-1e2e-4cc5-8fa2-ff2d38071a9c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:06:29.642192 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5316bb66-1e2e-4cc5-8fa2-ff2d38071a9c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:29.643642 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5316bb66-1e2e-4cc5-8fa2-ff2d38071a9c None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:29.655749 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 194/400] 10.222.0.44 () {56 vars in 1111 bytes} [Wed Jun 29 11:06:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:29.655749 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8f7720-f5bf-4eec-aeef-c989d2ba2df2 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:29.655749 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8f7720-f5bf-4eec-aeef-c989d2ba2df2 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:29.655749 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8f7720-f5bf-4eec-aeef-c989d2ba2df2 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:29.655749 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 195/401] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:06:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:06:29.659245 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8f7720-f5bf-4eec-aeef-c989d2ba2df2 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:29.673054 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28005a4-abfa-46be-973c-3fc7a356847c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:29.674525 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28005a4-abfa-46be-973c-3fc7a356847c None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:29.674986 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28005a4-abfa-46be-973c-3fc7a356847c None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:29.675812 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28005a4-abfa-46be-973c-3fc7a356847c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:29.689688 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28005a4-abfa-46be-973c-3fc7a356847c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 11:06:29.701151 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-b28005a4-abfa-46be-973c-3fc7a356847c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 11:06:29.701151 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:06:29.701151 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:06:29.701151 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:06:29.701151 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:06:29.701151 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:06:29.701151 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:06:29.701151 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:06:29.701151 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:06:29.701151 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:06:29.701151 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:06:29.701151 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:06:29.701151 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 11:06:29.701151 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 11:06:29.701151 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 11:06:29.701151 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 11:06:29.702134 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 11:06:29.702134 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 11:06:29.702134 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 11:06:29.702134 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 11:06:29.702134 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:06:29.702134 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:06:29.702134 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:06:29.702134 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:06:29.702134 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:06:29.702134 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:06:29.702134 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:06:29.702134 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jun 29 11:06:29.702134 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:06:29.702134 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:06:29.702134 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:06:29.702134 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:06:29.702930 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:06:29.702930 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:06:29.702930 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:06:29.702930 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:06:29.702930 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 11:06:29.702930 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 11:06:29.702930 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 11:06:29.702930 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 11:06:29.702930 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 11:06:29.702930 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 11:06:29.702930 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 11:06:29.702930 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jun 29 11:06:29.707587 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 207/402] 10.222.0.44 () {56 vars in 1096 bytes} [Wed Jun 29 11:06:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 11:06:29.710046 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d14e1a-7bdf-471b-a90d-94c707cc5ba8 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:29.710337 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d14e1a-7bdf-471b-a90d-94c707cc5ba8 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:29.710617 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d14e1a-7bdf-471b-a90d-94c707cc5ba8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:29.724143 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:29.727511 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-69d14e1a-7bdf-471b-a90d-94c707cc5ba8 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:06:29.728668 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-69d14e1a-7bdf-471b-a90d-94c707cc5ba8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:06:29.805269 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 208/403] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:06:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 11:06:29.806245 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d14e1a-7bdf-471b-a90d-94c707cc5ba8 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:29.807864 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8f7720-f5bf-4eec-aeef-c989d2ba2df2 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:29.845824 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00dd4d5-d833-4796-b0bf-fd09e4c7ebe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:29.847080 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00dd4d5-d833-4796-b0bf-fd09e4c7ebe5 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:29.847563 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00dd4d5-d833-4796-b0bf-fd09e4c7ebe5 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:29.848086 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00dd4d5-d833-4796-b0bf-fd09e4c7ebe5 None admin] PATH_INFO: `/v3/groups/8ec7aa0c6a3d47889a0b287a0fc483ad` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:29.849190 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:29.850242 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00dd4d5-d833-4796-b0bf-fd09e4c7ebe5 None admin] RBAC: Authorizing `identity:get_group(group_id=8ec7aa0c6a3d47889a0b287a0fc483ad)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:29.851911 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00dd4d5-d833-4796-b0bf-fd09e4c7ebe5 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:29.854030 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 196/404] 10.222.0.44 () {56 vars in 1180 bytes} [Wed Jun 29 11:06:29 2022] GET /identity/v3/groups/8ec7aa0c6a3d47889a0b287a0fc483ad => generated 226 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 11:06:29.854964 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00dd4d5-d833-4796-b0bf-fd09e4c7ebe5 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:29.915323 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d14e1a-7bdf-471b-a90d-94c707cc5ba8 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:29.953434 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9e6492-e702-45c1-ac36-b784c0e119d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:29.954027 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9e6492-e702-45c1-ac36-b784c0e119d6 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:29.954307 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9e6492-e702-45c1-ac36-b784c0e119d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:29.954790 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9e6492-e702-45c1-ac36-b784c0e119d6 None admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:29.955718 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9e6492-e702-45c1-ac36-b784c0e119d6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:06:29.956249 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9e6492-e702-45c1-ac36-b784c0e119d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:29.957804 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9e6492-e702-45c1-ac36-b784c0e119d6 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:29.964272 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00dd4d5-d833-4796-b0bf-fd09e4c7ebe5 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:29.966938 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 209/405] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Jun 29 11:06:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:29.967951 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9e6492-e702-45c1-ac36-b784c0e119d6 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:29.977193 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:30.007341 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14afb0f0-22ae-4e00-b3d9-9bee78a2a39b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:30.008751 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14afb0f0-22ae-4e00-b3d9-9bee78a2a39b None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:30.009265 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14afb0f0-22ae-4e00-b3d9-9bee78a2a39b None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:30.009736 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14afb0f0-22ae-4e00-b3d9-9bee78a2a39b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:30.022088 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14afb0f0-22ae-4e00-b3d9-9bee78a2a39b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 11:06:30.031077 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-14afb0f0-22ae-4e00-b3d9-9bee78a2a39b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 29 11:06:30.031077 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:06:30.031077 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:06:30.031077 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:06:30.031077 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:06:30.031077 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:06:30.031077 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:06:30.031077 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:06:30.031077 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:06:30.031077 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:06:30.031077 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:06:30.031077 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:06:30.031077 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 11:06:30.031077 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 11:06:30.031077 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 11:06:30.031077 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 11:06:30.032289 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 11:06:30.032289 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 11:06:30.032289 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 11:06:30.032289 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 11:06:30.032289 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:06:30.032289 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:06:30.032289 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:06:30.032289 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:06:30.032289 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:06:30.032289 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:06:30.032289 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:06:30.032289 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jun 29 11:06:30.032289 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:06:30.032289 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:06:30.032289 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:06:30.032289 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:06:30.033275 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:06:30.033275 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:06:30.033275 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:06:30.033275 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:06:30.033275 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 11:06:30.033275 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 11:06:30.033275 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 11:06:30.033275 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 11:06:30.033275 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 11:06:30.033275 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 11:06:30.033275 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 29 11:06:30.033275 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:06:30.038639 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 197/406] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Jun 29 11:06:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:06:30.040330 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14afb0f0-22ae-4e00-b3d9-9bee78a2a39b None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:30.076424 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9e6492-e702-45c1-ac36-b784c0e119d6 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:30.077899 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:30.111918 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c5242a-3b6d-4456-a0a2-d1f63a208680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:30.112941 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c5242a-3b6d-4456-a0a2-d1f63a208680 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:30.113331 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c5242a-3b6d-4456-a0a2-d1f63a208680 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:30.113712 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c5242a-3b6d-4456-a0a2-d1f63a208680 None admin] PATH_INFO: `/v3/projects/d36c6c9eb3234e82bb6a8537851eb336` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:30.115062 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c5242a-3b6d-4456-a0a2-d1f63a208680 None admin] RBAC: Authorizing `identity:get_project(project_id=d36c6c9eb3234e82bb6a8537851eb336)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:30.117024 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c5242a-3b6d-4456-a0a2-d1f63a208680 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:30.118611 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 210/407] 10.222.0.44 () {56 vars in 1186 bytes} [Wed Jun 29 11:06:29 2022] GET /identity/v3/projects/d36c6c9eb3234e82bb6a8537851eb336 => generated 302 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:30.124008 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c5242a-3b6d-4456-a0a2-d1f63a208680 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:30.156636 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14afb0f0-22ae-4e00-b3d9-9bee78a2a39b None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:30.198948 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c221de6c-a14a-44d7-bb03-80b2dff8528b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:30.200265 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c221de6c-a14a-44d7-bb03-80b2dff8528b None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:30.200745 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c221de6c-a14a-44d7-bb03-80b2dff8528b None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:30.201205 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c221de6c-a14a-44d7-bb03-80b2dff8528b None admin] PATH_INFO: `/v3/groups/9451dcb3c55545be80b6512695c19e9b` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:30.202582 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c221de6c-a14a-44d7-bb03-80b2dff8528b None admin] RBAC: Authorizing `identity:get_group(group_id=9451dcb3c55545be80b6512695c19e9b)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:30.204371 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c221de6c-a14a-44d7-bb03-80b2dff8528b None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:30.211947 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 198/408] 10.222.0.44 () {56 vars in 1180 bytes} [Wed Jun 29 11:06:30 2022] GET /identity/v3/groups/9451dcb3c55545be80b6512695c19e9b => generated 229 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:30.213126 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c221de6c-a14a-44d7-bb03-80b2dff8528b None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:30.231535 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c5242a-3b6d-4456-a0a2-d1f63a208680 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:30.256368 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:30.267474 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73e18c4-abf3-48d0-9e38-262528e6e977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:30.286586 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73e18c4-abf3-48d0-9e38-262528e6e977 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:30.286586 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73e18c4-abf3-48d0-9e38-262528e6e977 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:30.286586 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73e18c4-abf3-48d0-9e38-262528e6e977 None admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:30.286586 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73e18c4-abf3-48d0-9e38-262528e6e977 None admin] RBAC: Adding query filter params (name=member) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:06:30.286586 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73e18c4-abf3-48d0-9e38-262528e6e977 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:30.286586 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73e18c4-abf3-48d0-9e38-262528e6e977 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:30.287055 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 211/409] 10.222.0.44 () {56 vars in 1101 bytes} [Wed Jun 29 11:06:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:30.287055 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73e18c4-abf3-48d0-9e38-262528e6e977 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:30.313008 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:30.332091 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c221de6c-a14a-44d7-bb03-80b2dff8528b None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:30.370317 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3936a2-1b05-4b92-a823-a568001a08c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:30.371704 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3936a2-1b05-4b92-a823-a568001a08c3 None admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:30.372224 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3936a2-1b05-4b92-a823-a568001a08c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:30.372686 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3936a2-1b05-4b92-a823-a568001a08c3 None admin] PATH_INFO: `/v3/projects/d36c6c9eb3234e82bb6a8537851eb336/groups/8ec7aa0c6a3d47889a0b287a0fc483ad/roles/fc91ce4ad4a44cc3b05d0936754e87ec` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:30.375248 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3936a2-1b05-4b92-a823-a568001a08c3 None admin] RBAC: Authorizing `identity:create_grant(project_id=d36c6c9eb3234e82bb6a8537851eb336, group_id=8ec7aa0c6a3d47889a0b287a0fc483ad, role_id=fc91ce4ad4a44cc3b05d0936754e87ec)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:30.377128 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3936a2-1b05-4b92-a823-a568001a08c3 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:30.384201 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73e18c4-abf3-48d0-9e38-262528e6e977 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:30.419016 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02dab07-e3a5-41f4-a853-7325b5c28036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:30.419909 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02dab07-e3a5-41f4-a853-7325b5c28036 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:30.420208 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02dab07-e3a5-41f4-a853-7325b5c28036 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:30.420480 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02dab07-e3a5-41f4-a853-7325b5c28036 None admin] PATH_INFO: `/v3/projects/37897f6664b94d219c11b87aa269041a` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:30.421799 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02dab07-e3a5-41f4-a853-7325b5c28036 None admin] RBAC: Authorizing `identity:get_project(project_id=37897f6664b94d219c11b87aa269041a)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:30.423247 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02dab07-e3a5-41f4-a853-7325b5c28036 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:30.430353 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 212/410] 10.222.0.44 () {56 vars in 1186 bytes} [Wed Jun 29 11:06:30 2022] GET /identity/v3/projects/37897f6664b94d219c11b87aa269041a => generated 344 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:30.430353 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02dab07-e3a5-41f4-a853-7325b5c28036 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:30.488365 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 199/411] 10.222.0.44 () {58 vars in 1442 bytes} [Wed Jun 29 11:06:30 2022] PUT /identity/v3/projects/d36c6c9eb3234e82bb6a8537851eb336/groups/8ec7aa0c6a3d47889a0b287a0fc483ad/roles/fc91ce4ad4a44cc3b05d0936754e87ec => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:06:30.490642 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3936a2-1b05-4b92-a823-a568001a08c3 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:30.580449 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:30.584109 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02dab07-e3a5-41f4-a853-7325b5c28036 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:30.621084 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498090db-c77f-4ada-96ff-ffdabc3c4f1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:30.622254 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498090db-c77f-4ada-96ff-ffdabc3c4f1a None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:30.622689 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498090db-c77f-4ada-96ff-ffdabc3c4f1a None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:30.623117 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498090db-c77f-4ada-96ff-ffdabc3c4f1a None admin] PATH_INFO: `/v3/groups/8ec7aa0c6a3d47889a0b287a0fc483ad` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:30.624635 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498090db-c77f-4ada-96ff-ffdabc3c4f1a None admin] RBAC: Authorizing `identity:get_group(group_id=8ec7aa0c6a3d47889a0b287a0fc483ad)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:30.626317 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498090db-c77f-4ada-96ff-ffdabc3c4f1a None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:30.628615 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 213/412] 10.222.0.44 () {56 vars in 1180 bytes} [Wed Jun 29 11:06:30 2022] GET /identity/v3/groups/8ec7aa0c6a3d47889a0b287a0fc483ad => generated 226 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:30.630222 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebcb1ef-f528-48ef-a2f6-2bc6ab6098a0 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:30.630701 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebcb1ef-f528-48ef-a2f6-2bc6ab6098a0 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:30.631118 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebcb1ef-f528-48ef-a2f6-2bc6ab6098a0 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:30.636275 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 214/413] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:06:30 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:06:30.637394 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebcb1ef-f528-48ef-a2f6-2bc6ab6098a0 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:30.657363 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:30.666277 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3936a2-1b05-4b92-a823-a568001a08c3 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:30.712546 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f28d9fd-0d9f-4a89-a46a-c018f1255eb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:30.713550 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f28d9fd-0d9f-4a89-a46a-c018f1255eb0 None admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:30.713895 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f28d9fd-0d9f-4a89-a46a-c018f1255eb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:30.714247 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f28d9fd-0d9f-4a89-a46a-c018f1255eb0 None admin] PATH_INFO: `/v3/projects/37897f6664b94d219c11b87aa269041a/groups/9451dcb3c55545be80b6512695c19e9b/roles/e95c2ce63e114cc19a9dda6ad21e794f` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:30.716602 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f28d9fd-0d9f-4a89-a46a-c018f1255eb0 None admin] RBAC: Authorizing `identity:create_grant(project_id=37897f6664b94d219c11b87aa269041a, group_id=9451dcb3c55545be80b6512695c19e9b, role_id=e95c2ce63e114cc19a9dda6ad21e794f)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:30.718137 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f28d9fd-0d9f-4a89-a46a-c018f1255eb0 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:30.763558 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebcb1ef-f528-48ef-a2f6-2bc6ab6098a0 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:30.797884 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52103b7f-bb92-4e1a-9652-bcd4700f96a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:30.799105 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52103b7f-bb92-4e1a-9652-bcd4700f96a8 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:30.799563 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52103b7f-bb92-4e1a-9652-bcd4700f96a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:30.800041 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52103b7f-bb92-4e1a-9652-bcd4700f96a8 None admin] PATH_INFO: `/v3/projects/d36c6c9eb3234e82bb6a8537851eb336` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:30.801408 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52103b7f-bb92-4e1a-9652-bcd4700f96a8 None admin] RBAC: Authorizing `identity:get_project(project_id=d36c6c9eb3234e82bb6a8537851eb336)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:30.803093 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52103b7f-bb92-4e1a-9652-bcd4700f96a8 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:30.805372 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 215/414] 10.222.0.44 () {56 vars in 1186 bytes} [Wed Jun 29 11:06:30 2022] GET /identity/v3/projects/d36c6c9eb3234e82bb6a8537851eb336 => generated 302 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:30.806758 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d379c5-c2df-4537-a09b-087d5da4dd04 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:30.807206 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d379c5-c2df-4537-a09b-087d5da4dd04 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:30.807644 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d379c5-c2df-4537-a09b-087d5da4dd04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:30.826049 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-68d379c5-c2df-4537-a09b-087d5da4dd04 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:06:30.827735 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-68d379c5-c2df-4537-a09b-087d5da4dd04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:06:30.846836 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 200/415] 10.222.0.44 () {58 vars in 1442 bytes} [Wed Jun 29 11:06:30 2022] PUT /identity/v3/projects/37897f6664b94d219c11b87aa269041a/groups/9451dcb3c55545be80b6512695c19e9b/roles/e95c2ce63e114cc19a9dda6ad21e794f => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:06:30.849512 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f28d9fd-0d9f-4a89-a46a-c018f1255eb0 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:30.882699 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:30.981062 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 216/416] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:06:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 11:06:31.007747 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d379c5-c2df-4537-a09b-087d5da4dd04 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:31.054309 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f28d9fd-0d9f-4a89-a46a-c018f1255eb0 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:31.104032 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7ffab4-4c16-473d-809b-549183de4f37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:31.104914 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7ffab4-4c16-473d-809b-549183de4f37 None admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:31.105195 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7ffab4-4c16-473d-809b-549183de4f37 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:31.105465 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7ffab4-4c16-473d-809b-549183de4f37 None admin] PATH_INFO: `/v3/projects/d36c6c9eb3234e82bb6a8537851eb336/groups/8ec7aa0c6a3d47889a0b287a0fc483ad/roles/e1c88eebe911432cab6a07fbeb84653b` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:31.107511 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7ffab4-4c16-473d-809b-549183de4f37 None admin] RBAC: Authorizing `identity:create_grant(project_id=d36c6c9eb3234e82bb6a8537851eb336, group_id=8ec7aa0c6a3d47889a0b287a0fc483ad, role_id=e1c88eebe911432cab6a07fbeb84653b)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:31.109527 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7ffab4-4c16-473d-809b-549183de4f37 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:31.136959 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d379c5-c2df-4537-a09b-087d5da4dd04 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:31.172373 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9251cbba-c072-4b4a-b215-c51c70647cc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:31.173186 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9251cbba-c072-4b4a-b215-c51c70647cc1 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:31.173544 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9251cbba-c072-4b4a-b215-c51c70647cc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:31.173762 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9251cbba-c072-4b4a-b215-c51c70647cc1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:31.179005 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 201/417] 10.222.0.44 () {58 vars in 1442 bytes} [Wed Jun 29 11:06:30 2022] PUT /identity/v3/projects/d36c6c9eb3234e82bb6a8537851eb336/groups/8ec7aa0c6a3d47889a0b287a0fc483ad/roles/e1c88eebe911432cab6a07fbeb84653b => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:06:31.181345 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6096a71d-d2c3-41c8-b507-cd788098f9b5 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:31.181926 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6096a71d-d2c3-41c8-b507-cd788098f9b5 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:31.186905 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6096a71d-d2c3-41c8-b507-cd788098f9b5 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:31.186905 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 202/418] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:06:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:06:31.189398 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739fb122-f4d9-48a3-a793-395d17d82267 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:31.189790 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739fb122-f4d9-48a3-a793-395d17d82267 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:31.190629 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739fb122-f4d9-48a3-a793-395d17d82267 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:31.196239 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9251cbba-c072-4b4a-b215-c51c70647cc1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 11:06:31.204659 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-9251cbba-c072-4b4a-b215-c51c70647cc1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 29 11:06:31.204659 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:06:31.204659 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:06:31.204659 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:06:31.204659 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:06:31.204659 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:06:31.204659 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:06:31.204659 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:06:31.204659 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:06:31.204659 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:06:31.204659 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:06:31.204659 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:06:31.204659 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 11:06:31.204659 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 11:06:31.204659 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 11:06:31.204659 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 11:06:31.205580 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 11:06:31.205580 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 11:06:31.205580 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 11:06:31.205580 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 11:06:31.205580 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:06:31.205580 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:06:31.205580 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:06:31.205580 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:06:31.205580 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:06:31.205580 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:06:31.205580 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:06:31.205580 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jun 29 11:06:31.205580 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:06:31.205580 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:06:31.205580 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:06:31.205580 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:06:31.206406 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:06:31.206406 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:06:31.206406 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:06:31.206406 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:06:31.206406 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 11:06:31.206406 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 11:06:31.206406 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 11:06:31.206406 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 11:06:31.206406 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 11:06:31.206406 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 11:06:31.206406 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 29 11:06:31.206406 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jun 29 11:06:31.209940 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-739fb122-f4d9-48a3-a793-395d17d82267 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:06:31.211240 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-739fb122-f4d9-48a3-a793-395d17d82267 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:06:31.212393 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 217/419] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Jun 29 11:06:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:06:31.218269 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9251cbba-c072-4b4a-b215-c51c70647cc1 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:31.232551 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:31.290013 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:31.297269 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 203/420] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:06:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 11:06:31.333225 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739fb122-f4d9-48a3-a793-395d17d82267 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:31.385670 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9251cbba-c072-4b4a-b215-c51c70647cc1 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:31.429308 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034ceedb-7e75-49b2-a783-6563086fe717 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:31.429992 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034ceedb-7e75-49b2-a783-6563086fe717 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:31.430284 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034ceedb-7e75-49b2-a783-6563086fe717 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:31.430563 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034ceedb-7e75-49b2-a783-6563086fe717 None admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:31.431385 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034ceedb-7e75-49b2-a783-6563086fe717 None admin] RBAC: Adding query filter params (name=member) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:06:31.431698 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034ceedb-7e75-49b2-a783-6563086fe717 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:31.433430 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034ceedb-7e75-49b2-a783-6563086fe717 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:31.447428 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 218/421] 10.222.0.44 () {56 vars in 1101 bytes} [Wed Jun 29 11:06:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 11:06:31.448710 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034ceedb-7e75-49b2-a783-6563086fe717 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:31.513060 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739fb122-f4d9-48a3-a793-395d17d82267 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:31.557675 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11fd30b-422a-404a-8dad-7fd865bf825b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:31.558491 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11fd30b-422a-404a-8dad-7fd865bf825b None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:31.558773 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11fd30b-422a-404a-8dad-7fd865bf825b None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:31.559045 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11fd30b-422a-404a-8dad-7fd865bf825b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:31.578395 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034ceedb-7e75-49b2-a783-6563086fe717 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:31.580151 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:31.597478 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11fd30b-422a-404a-8dad-7fd865bf825b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 11:06:31.602540 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-e11fd30b-422a-404a-8dad-7fd865bf825b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 29 11:06:31.602540 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:06:31.602540 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:06:31.602540 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:06:31.602540 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:06:31.602540 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:06:31.602540 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:06:31.602540 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:06:31.602540 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:06:31.602540 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:06:31.602540 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:06:31.602540 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:06:31.602540 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 11:06:31.602540 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 11:06:31.602540 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 11:06:31.602540 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 11:06:31.603452 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 11:06:31.603452 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 11:06:31.603452 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 11:06:31.603452 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 11:06:31.603452 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:06:31.603452 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:06:31.603452 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:06:31.603452 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:06:31.603452 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:06:31.603452 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:06:31.603452 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:06:31.603452 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jun 29 11:06:31.603452 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:06:31.603452 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:06:31.603452 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:06:31.603452 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:06:31.604288 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:06:31.604288 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:06:31.604288 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:06:31.604288 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:06:31.604288 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 11:06:31.604288 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 11:06:31.604288 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 11:06:31.604288 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 11:06:31.604288 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 11:06:31.604288 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 11:06:31.604288 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 29 11:06:31.604288 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:06:31.616376 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 204/422] 10.222.0.44 () {56 vars in 1114 bytes} [Wed Jun 29 11:06:31 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:06:31.617325 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11fd30b-422a-404a-8dad-7fd865bf825b None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:31.625514 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:31.654411 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47d7a97-286b-4573-8729-d51c9fb49140 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:31.655066 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47d7a97-286b-4573-8729-d51c9fb49140 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:31.655347 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47d7a97-286b-4573-8729-d51c9fb49140 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:31.655638 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47d7a97-286b-4573-8729-d51c9fb49140 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:31.748775 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11fd30b-422a-404a-8dad-7fd865bf825b None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:31.791821 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afb659e-b6ec-4ecd-a634-9b47e6d2a953 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:31.792571 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afb659e-b6ec-4ecd-a634-9b47e6d2a953 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:31.792857 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afb659e-b6ec-4ecd-a634-9b47e6d2a953 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:31.793205 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afb659e-b6ec-4ecd-a634-9b47e6d2a953 None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:31.794565 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afb659e-b6ec-4ecd-a634-9b47e6d2a953 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:06:31.795000 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afb659e-b6ec-4ecd-a634-9b47e6d2a953 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:31.798425 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afb659e-b6ec-4ecd-a634-9b47e6d2a953 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:31.806800 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 205/423] 10.222.0.44 () {56 vars in 1111 bytes} [Wed Jun 29 11:06:31 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:31.812913 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afb659e-b6ec-4ecd-a634-9b47e6d2a953 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:31.844669 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47d7a97-286b-4573-8729-d51c9fb49140 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:31.846145 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47d7a97-286b-4573-8729-d51c9fb49140 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:31.888317 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:31.932756 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afb659e-b6ec-4ecd-a634-9b47e6d2a953 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:31.970558 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f2726d-4af8-4a4b-91a5-258916136f65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:31.971205 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f2726d-4af8-4a4b-91a5-258916136f65 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:31.971488 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f2726d-4af8-4a4b-91a5-258916136f65 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:31.971767 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f2726d-4af8-4a4b-91a5-258916136f65 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:32.044085 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f2726d-4af8-4a4b-91a5-258916136f65 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:32.045779 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f2726d-4af8-4a4b-91a5-258916136f65 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:32.120246 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:32.126060 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:32.149010 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 219/424] 10.222.0.44 () {58 vars in 1282 bytes} [Wed Jun 29 11:06:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 700 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 29 11:06:32.154154 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47d7a97-286b-4573-8729-d51c9fb49140 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:32.155144 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 206/425] 10.222.0.44 () {58 vars in 1282 bytes} [Wed Jun 29 11:06:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 29 11:06:32.160622 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f2726d-4af8-4a4b-91a5-258916136f65 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:32.268892 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f2726d-4af8-4a4b-91a5-258916136f65 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:32.305687 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03594c99-65ed-413e-993c-887f50ef8873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:32.306541 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03594c99-65ed-413e-993c-887f50ef8873 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:32.306825 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03594c99-65ed-413e-993c-887f50ef8873 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:32.307100 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03594c99-65ed-413e-993c-887f50ef8873 None admin] PATH_INFO: `/v3/projects/3b98d22914de4e3b8bca7fe0a0b51357` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:32.308335 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03594c99-65ed-413e-993c-887f50ef8873 None admin] RBAC: Authorizing `identity:get_project(project_id=3b98d22914de4e3b8bca7fe0a0b51357)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:32.309758 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03594c99-65ed-413e-993c-887f50ef8873 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:32.312338 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 207/426] 10.222.0.44 () {56 vars in 1186 bytes} [Wed Jun 29 11:06:32 2022] GET /identity/v3/projects/3b98d22914de4e3b8bca7fe0a0b51357 => generated 298 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 11:06:32.317706 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03594c99-65ed-413e-993c-887f50ef8873 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:32.356607 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:32.358610 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47d7a97-286b-4573-8729-d51c9fb49140 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:32.425124 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03594c99-65ed-413e-993c-887f50ef8873 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:32.428530 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a349fc-7ed3-4ec2-ad05-9b70c8eb7da6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:32.429405 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a349fc-7ed3-4ec2-ad05-9b70c8eb7da6 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:32.429684 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a349fc-7ed3-4ec2-ad05-9b70c8eb7da6 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:32.429954 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a349fc-7ed3-4ec2-ad05-9b70c8eb7da6 None admin] PATH_INFO: `/v3/projects/3b98d22914de4e3b8bca7fe0a0b51357` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:32.431199 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a349fc-7ed3-4ec2-ad05-9b70c8eb7da6 None admin] RBAC: Authorizing `identity:get_project(project_id=3b98d22914de4e3b8bca7fe0a0b51357)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:32.432881 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a349fc-7ed3-4ec2-ad05-9b70c8eb7da6 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:32.435869 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 220/427] 10.222.0.44 () {56 vars in 1186 bytes} [Wed Jun 29 11:06:32 2022] GET /identity/v3/projects/3b98d22914de4e3b8bca7fe0a0b51357 => generated 298 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:32.440331 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a349fc-7ed3-4ec2-ad05-9b70c8eb7da6 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:32.512221 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:32.540945 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a349fc-7ed3-4ec2-ad05-9b70c8eb7da6 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:32.561900 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a7ff21-b943-4b09-82d0-fae13e1dbb7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:32.562758 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a7ff21-b943-4b09-82d0-fae13e1dbb7a None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:32.563071 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a7ff21-b943-4b09-82d0-fae13e1dbb7a None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:32.563341 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a7ff21-b943-4b09-82d0-fae13e1dbb7a None admin] PATH_INFO: `/v3/groups/8ec7aa0c6a3d47889a0b287a0fc483ad` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:32.568575 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a7ff21-b943-4b09-82d0-fae13e1dbb7a None admin] RBAC: Authorizing `identity:get_group(group_id=8ec7aa0c6a3d47889a0b287a0fc483ad)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:32.570033 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a7ff21-b943-4b09-82d0-fae13e1dbb7a None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:32.572267 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 208/428] 10.222.0.44 () {56 vars in 1180 bytes} [Wed Jun 29 11:06:32 2022] GET /identity/v3/groups/8ec7aa0c6a3d47889a0b287a0fc483ad => generated 226 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 11:06:32.576813 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e69651-a961-468e-a312-95a72612d966 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:32.578064 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e69651-a961-468e-a312-95a72612d966 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:32.578173 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a7ff21-b943-4b09-82d0-fae13e1dbb7a None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:32.578343 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e69651-a961-468e-a312-95a72612d966 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:32.578610 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e69651-a961-468e-a312-95a72612d966 None admin] PATH_INFO: `/v3/groups/8ec7aa0c6a3d47889a0b287a0fc483ad` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:32.579809 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e69651-a961-468e-a312-95a72612d966 None admin] RBAC: Authorizing `identity:get_group(group_id=8ec7aa0c6a3d47889a0b287a0fc483ad)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:32.581281 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e69651-a961-468e-a312-95a72612d966 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:32.588005 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 221/429] 10.222.0.44 () {56 vars in 1180 bytes} [Wed Jun 29 11:06:32 2022] GET /identity/v3/groups/8ec7aa0c6a3d47889a0b287a0fc483ad => generated 226 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 11:06:32.589370 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e69651-a961-468e-a312-95a72612d966 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:32.684642 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a7ff21-b943-4b09-82d0-fae13e1dbb7a None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:32.692272 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:32.701541 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e69651-a961-468e-a312-95a72612d966 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:32.720090 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dedfe9e-1a96-422d-b81a-cc9f52398fbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:32.720751 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dedfe9e-1a96-422d-b81a-cc9f52398fbe None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:32.721032 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dedfe9e-1a96-422d-b81a-cc9f52398fbe None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:32.721307 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dedfe9e-1a96-422d-b81a-cc9f52398fbe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:32.722135 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dedfe9e-1a96-422d-b81a-cc9f52398fbe None admin] RBAC: Adding query filter params (group.id=8ec7aa0c6a3d47889a0b287a0fc483ad, role.id=fc91ce4ad4a44cc3b05d0936754e87ec, scope.project.id=3b98d22914de4e3b8bca7fe0a0b51357) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:06:32.722501 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dedfe9e-1a96-422d-b81a-cc9f52398fbe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:32.723972 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dedfe9e-1a96-422d-b81a-cc9f52398fbe None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:32.730589 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 209/430] 10.222.0.44 () {56 vars in 1376 bytes} [Wed Jun 29 11:06:32 2022] GET /identity/v3/role_assignments?group.id=8ec7aa0c6a3d47889a0b287a0fc483ad&scope.project.id=3b98d22914de4e3b8bca7fe0a0b51357&role.id=fc91ce4ad4a44cc3b05d0936754e87ec => generated 611 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 11:06:32.748261 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:32.766647 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd713d2b-6f83-4e24-b7dc-1931321f30be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:32.767324 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd713d2b-6f83-4e24-b7dc-1931321f30be None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:32.767608 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd713d2b-6f83-4e24-b7dc-1931321f30be None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:32.767908 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd713d2b-6f83-4e24-b7dc-1931321f30be None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:32.768881 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd713d2b-6f83-4e24-b7dc-1931321f30be None admin] RBAC: Adding query filter params (group.id=8ec7aa0c6a3d47889a0b287a0fc483ad, role.id=e1c88eebe911432cab6a07fbeb84653b, scope.project.id=3b98d22914de4e3b8bca7fe0a0b51357) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:06:32.769195 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd713d2b-6f83-4e24-b7dc-1931321f30be None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:32.770669 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd713d2b-6f83-4e24-b7dc-1931321f30be None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:32.782553 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 222/431] 10.222.0.44 () {56 vars in 1376 bytes} [Wed Jun 29 11:06:32 2022] GET /identity/v3/role_assignments?group.id=8ec7aa0c6a3d47889a0b287a0fc483ad&scope.project.id=3b98d22914de4e3b8bca7fe0a0b51357&role.id=e1c88eebe911432cab6a07fbeb84653b => generated 611 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 11:06:32.841869 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b02cb55-9718-427a-b7ef-5f6973d309d9 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:32.842209 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b02cb55-9718-427a-b7ef-5f6973d309d9 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:32.842435 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b02cb55-9718-427a-b7ef-5f6973d309d9 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:32.843870 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 210/432] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:06:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:06:32.850192 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4884d853-0583-4175-88aa-55db4cfc4cbe None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:32.850486 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4884d853-0583-4175-88aa-55db4cfc4cbe None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:32.850745 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4884d853-0583-4175-88aa-55db4cfc4cbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:32.868397 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-4884d853-0583-4175-88aa-55db4cfc4cbe None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:06:32.869617 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-4884d853-0583-4175-88aa-55db4cfc4cbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:06:32.958168 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 223/433] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:06:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 29 11:06:32.984831 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b02cb55-9718-427a-b7ef-5f6973d309d9 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:33.097168 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b02cb55-9718-427a-b7ef-5f6973d309d9 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:33.140560 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b12f04e-82af-49da-9159-dc9ea6ccce0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:33.141371 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b12f04e-82af-49da-9159-dc9ea6ccce0a None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:33.141650 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b12f04e-82af-49da-9159-dc9ea6ccce0a None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:33.141922 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b12f04e-82af-49da-9159-dc9ea6ccce0a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:33.153289 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b12f04e-82af-49da-9159-dc9ea6ccce0a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 11:06:33.161484 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-8b12f04e-82af-49da-9159-dc9ea6ccce0a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 29 11:06:33.161484 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:06:33.161484 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:06:33.161484 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:06:33.161484 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:06:33.161484 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:06:33.161484 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:06:33.161484 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:06:33.161484 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:06:33.161484 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:06:33.161484 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:06:33.161484 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:06:33.161484 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 11:06:33.161484 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 11:06:33.161484 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 11:06:33.161484 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 11:06:33.162413 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 11:06:33.162413 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 11:06:33.162413 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 11:06:33.162413 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 11:06:33.162413 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:06:33.162413 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:06:33.162413 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:06:33.162413 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:06:33.162413 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:06:33.162413 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:06:33.162413 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:06:33.162413 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jun 29 11:06:33.162413 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:06:33.162413 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:06:33.162413 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:06:33.162413 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:06:33.163639 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:06:33.163639 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:06:33.163639 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:06:33.163639 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:06:33.163639 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 11:06:33.163639 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 11:06:33.163639 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 11:06:33.163639 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 11:06:33.163639 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 11:06:33.163639 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 11:06:33.163639 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 29 11:06:33.163639 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:06:33.163639 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 211/434] 10.222.0.44 () {56 vars in 1114 bytes} [Wed Jun 29 11:06:32 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:06:33.168679 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4884d853-0583-4175-88aa-55db4cfc4cbe None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:33.256192 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807db29f-e905-4c96-8597-e7adb7eb32a8 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:33.256788 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807db29f-e905-4c96-8597-e7adb7eb32a8 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:33.257386 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807db29f-e905-4c96-8597-e7adb7eb32a8 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:33.259262 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 212/435] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:06:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:06:33.264444 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae2b74c-a321-40b8-b973-63cfb486698f None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:33.264767 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae2b74c-a321-40b8-b973-63cfb486698f None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:33.265120 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae2b74c-a321-40b8-b973-63cfb486698f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:33.282236 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4884d853-0583-4175-88aa-55db4cfc4cbe None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:33.282444 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-cae2b74c-a321-40b8-b973-63cfb486698f None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:06:33.283637 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-cae2b74c-a321-40b8-b973-63cfb486698f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:06:33.304733 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:33.306394 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:33.329378 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4dc5c7-f2f1-4061-9143-1c0ae8506074 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:33.330047 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4dc5c7-f2f1-4061-9143-1c0ae8506074 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:33.330327 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4dc5c7-f2f1-4061-9143-1c0ae8506074 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:33.330597 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4dc5c7-f2f1-4061-9143-1c0ae8506074 None admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:33.331340 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4dc5c7-f2f1-4061-9143-1c0ae8506074 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:06:33.331641 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4dc5c7-f2f1-4061-9143-1c0ae8506074 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:33.333066 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4dc5c7-f2f1-4061-9143-1c0ae8506074 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:33.344574 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 224/436] 10.222.0.44 () {56 vars in 1111 bytes} [Wed Jun 29 11:06:33 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:33.351470 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4dc5c7-f2f1-4061-9143-1c0ae8506074 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:33.372785 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 213/437] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:06:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 29 11:06:33.397020 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae2b74c-a321-40b8-b973-63cfb486698f None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:33.479660 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4dc5c7-f2f1-4061-9143-1c0ae8506074 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:33.523503 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92226557-d6df-4c8a-b555-c42cb8107eeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:33.524121 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92226557-d6df-4c8a-b555-c42cb8107eeb None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:33.524407 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92226557-d6df-4c8a-b555-c42cb8107eeb None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:33.524681 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92226557-d6df-4c8a-b555-c42cb8107eeb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:33.541251 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae2b74c-a321-40b8-b973-63cfb486698f None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:33.586312 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:33.592964 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bf73bf-4a11-401f-a949-b979539a9fe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:33.593765 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bf73bf-4a11-401f-a949-b979539a9fe3 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:33.594076 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bf73bf-4a11-401f-a949-b979539a9fe3 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:33.601162 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bf73bf-4a11-401f-a949-b979539a9fe3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:33.606504 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bf73bf-4a11-401f-a949-b979539a9fe3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 11:06:33.608996 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92226557-d6df-4c8a-b555-c42cb8107eeb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:33.610466 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92226557-d6df-4c8a-b555-c42cb8107eeb None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:33.615995 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-d0bf73bf-4a11-401f-a949-b979539a9fe3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 11:06:33.615995 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:06:33.615995 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:06:33.615995 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:06:33.615995 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:06:33.615995 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:06:33.615995 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:06:33.615995 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:06:33.615995 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:06:33.615995 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:06:33.615995 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:06:33.615995 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:06:33.615995 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 11:06:33.615995 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 11:06:33.615995 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 11:06:33.615995 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 11:06:33.616871 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 11:06:33.616871 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 11:06:33.616871 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 11:06:33.616871 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 11:06:33.616871 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:06:33.616871 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:06:33.616871 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:06:33.616871 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:06:33.616871 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:06:33.616871 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:06:33.616871 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:06:33.616871 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jun 29 11:06:33.616871 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:06:33.616871 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:06:33.616871 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:06:33.616871 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:06:33.617654 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:06:33.617654 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:06:33.617654 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:06:33.617654 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:06:33.617654 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 11:06:33.617654 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 11:06:33.617654 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 11:06:33.617654 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 11:06:33.617654 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 11:06:33.617654 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 11:06:33.617654 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 11:06:33.617654 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:06:33.620383 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 214/438] 10.222.0.44 () {56 vars in 1096 bytes} [Wed Jun 29 11:06:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:06:33.627584 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bf73bf-4a11-401f-a949-b979539a9fe3 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:33.652197 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:33.728669 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 225/439] 10.222.0.44 () {58 vars in 1282 bytes} [Wed Jun 29 11:06:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 29 11:06:33.733820 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92226557-d6df-4c8a-b555-c42cb8107eeb None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:33.746973 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bf73bf-4a11-401f-a949-b979539a9fe3 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:33.782222 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbbc1e0-5a0e-4c21-9eff-893ee06f1d9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:33.782815 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbbc1e0-5a0e-4c21-9eff-893ee06f1d9f None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:33.783097 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbbc1e0-5a0e-4c21-9eff-893ee06f1d9f None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:33.783387 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbbc1e0-5a0e-4c21-9eff-893ee06f1d9f None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:33.784169 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbbc1e0-5a0e-4c21-9eff-893ee06f1d9f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:06:33.784480 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbbc1e0-5a0e-4c21-9eff-893ee06f1d9f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:33.785864 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbbc1e0-5a0e-4c21-9eff-893ee06f1d9f None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:33.798844 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 215/440] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Jun 29 11:06:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 11:06:33.800073 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbbc1e0-5a0e-4c21-9eff-893ee06f1d9f None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:33.824732 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:33.843246 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92226557-d6df-4c8a-b555-c42cb8107eeb None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:33.875629 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbf12b4-9336-4e3e-84e0-f4f586576d1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:33.876519 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbf12b4-9336-4e3e-84e0-f4f586576d1d None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:33.876797 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbf12b4-9336-4e3e-84e0-f4f586576d1d None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:33.877069 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbf12b4-9336-4e3e-84e0-f4f586576d1d None admin] PATH_INFO: `/v3/projects/d36c6c9eb3234e82bb6a8537851eb336` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:33.878251 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbf12b4-9336-4e3e-84e0-f4f586576d1d None admin] RBAC: Authorizing `identity:get_project(project_id=d36c6c9eb3234e82bb6a8537851eb336)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:33.879537 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:33.879717 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbf12b4-9336-4e3e-84e0-f4f586576d1d None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:33.881955 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 226/441] 10.222.0.44 () {56 vars in 1186 bytes} [Wed Jun 29 11:06:33 2022] GET /identity/v3/projects/d36c6c9eb3234e82bb6a8537851eb336 => generated 302 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 11:06:33.883105 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15949d8f-658a-4ad4-b1c2-52aff8d7319a None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:33.883392 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15949d8f-658a-4ad4-b1c2-52aff8d7319a None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:33.883718 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15949d8f-658a-4ad4-b1c2-52aff8d7319a None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:33.885113 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 227/442] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:06:33 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:06:33.887214 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15949d8f-658a-4ad4-b1c2-52aff8d7319a None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:33.901536 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbbc1e0-5a0e-4c21-9eff-893ee06f1d9f None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:33.935565 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da4fdb8-9e74-4e13-9945-5e93c7d75801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:33.936238 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da4fdb8-9e74-4e13-9945-5e93c7d75801 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:33.936526 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da4fdb8-9e74-4e13-9945-5e93c7d75801 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:33.936794 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da4fdb8-9e74-4e13-9945-5e93c7d75801 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:33.990921 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15949d8f-658a-4ad4-b1c2-52aff8d7319a None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:34.003271 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da4fdb8-9e74-4e13-9945-5e93c7d75801 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:34.004892 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da4fdb8-9e74-4e13-9945-5e93c7d75801 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:34.024732 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a08aeac-5116-4310-9cf2-8cf0f093d118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:34.025636 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a08aeac-5116-4310-9cf2-8cf0f093d118 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:34.025922 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a08aeac-5116-4310-9cf2-8cf0f093d118 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:34.026195 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a08aeac-5116-4310-9cf2-8cf0f093d118 None admin] PATH_INFO: `/v3/groups/8ec7aa0c6a3d47889a0b287a0fc483ad` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:34.027375 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a08aeac-5116-4310-9cf2-8cf0f093d118 None admin] RBAC: Authorizing `identity:get_group(group_id=8ec7aa0c6a3d47889a0b287a0fc483ad)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:34.028927 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a08aeac-5116-4310-9cf2-8cf0f093d118 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:34.030587 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 228/443] 10.222.0.44 () {56 vars in 1180 bytes} [Wed Jun 29 11:06:33 2022] GET /identity/v3/groups/8ec7aa0c6a3d47889a0b287a0fc483ad => generated 226 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 11:06:34.033250 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcd291e-235d-4c63-8af4-5caa02be9cc8 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:34.033467 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcd291e-235d-4c63-8af4-5caa02be9cc8 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:34.033821 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcd291e-235d-4c63-8af4-5caa02be9cc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:34.050217 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-bfcd291e-235d-4c63-8af4-5caa02be9cc8 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:06:34.051417 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-bfcd291e-235d-4c63-8af4-5caa02be9cc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:06:34.062552 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:34.090534 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:34.111856 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 216/444] 10.222.0.44 () {58 vars in 1282 bytes} [Wed Jun 29 11:06:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 29 11:06:34.116747 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da4fdb8-9e74-4e13-9945-5e93c7d75801 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:34.119001 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 229/445] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:06:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 11:06:34.119920 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcd291e-235d-4c63-8af4-5caa02be9cc8 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:34.215053 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da4fdb8-9e74-4e13-9945-5e93c7d75801 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:34.216391 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcd291e-235d-4c63-8af4-5caa02be9cc8 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:34.248877 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075a28d3-553c-4242-949d-afffe0b47047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:34.249517 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075a28d3-553c-4242-949d-afffe0b47047 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:34.249798 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075a28d3-553c-4242-949d-afffe0b47047 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:34.250067 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075a28d3-553c-4242-949d-afffe0b47047 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:34.250179 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352172d5-638f-4dc1-ab22-e5122e8817a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:34.250886 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075a28d3-553c-4242-949d-afffe0b47047 None admin] RBAC: Adding query filter params (group.id=8ec7aa0c6a3d47889a0b287a0fc483ad, role.id=fc91ce4ad4a44cc3b05d0936754e87ec, scope.project.id=d36c6c9eb3234e82bb6a8537851eb336) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:06:34.251028 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352172d5-638f-4dc1-ab22-e5122e8817a1 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:34.251193 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075a28d3-553c-4242-949d-afffe0b47047 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:34.251313 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352172d5-638f-4dc1-ab22-e5122e8817a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:34.251587 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352172d5-638f-4dc1-ab22-e5122e8817a1 None admin] PATH_INFO: `/v3/projects/37897f6664b94d219c11b87aa269041a` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:34.252712 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075a28d3-553c-4242-949d-afffe0b47047 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:34.252803 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352172d5-638f-4dc1-ab22-e5122e8817a1 None admin] RBAC: Authorizing `identity:get_project(project_id=37897f6664b94d219c11b87aa269041a)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:34.254229 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352172d5-638f-4dc1-ab22-e5122e8817a1 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:34.256567 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 230/446] 10.222.0.44 () {56 vars in 1186 bytes} [Wed Jun 29 11:06:34 2022] GET /identity/v3/projects/37897f6664b94d219c11b87aa269041a => generated 344 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:34.257324 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352172d5-638f-4dc1-ab22-e5122e8817a1 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:34.259865 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 217/447] 10.222.0.44 () {56 vars in 1376 bytes} [Wed Jun 29 11:06:34 2022] GET /identity/v3/role_assignments?group.id=8ec7aa0c6a3d47889a0b287a0fc483ad&scope.project.id=d36c6c9eb3234e82bb6a8537851eb336&role.id=fc91ce4ad4a44cc3b05d0936754e87ec => generated 611 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:34.261903 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075a28d3-553c-4242-949d-afffe0b47047 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:34.280662 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:34.312116 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:34.379034 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352172d5-638f-4dc1-ab22-e5122e8817a1 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:34.379295 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075a28d3-553c-4242-949d-afffe0b47047 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:34.419417 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baeeeabe-2781-4d6f-a851-c385d1f722f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:34.420185 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeeeabe-2781-4d6f-a851-c385d1f722f6 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:34.421169 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeeeabe-2781-4d6f-a851-c385d1f722f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:34.421169 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeeeabe-2781-4d6f-a851-c385d1f722f6 None admin] PATH_INFO: `/v3/groups/9451dcb3c55545be80b6512695c19e9b` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:34.421554 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572157e0-a314-46e1-b5ca-b79296dd99ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:34.422143 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baeeeabe-2781-4d6f-a851-c385d1f722f6 None admin] RBAC: Authorizing `identity:get_group(group_id=9451dcb3c55545be80b6512695c19e9b)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:34.422342 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572157e0-a314-46e1-b5ca-b79296dd99ac None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:34.422621 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572157e0-a314-46e1-b5ca-b79296dd99ac None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:34.422892 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572157e0-a314-46e1-b5ca-b79296dd99ac None admin] PATH_INFO: `/v3/roles/member` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:34.423573 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baeeeabe-2781-4d6f-a851-c385d1f722f6 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:34.425404 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 218/448] 10.222.0.44 () {56 vars in 1180 bytes} [Wed Jun 29 11:06:34 2022] GET /identity/v3/groups/9451dcb3c55545be80b6512695c19e9b => generated 229 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 11:06:34.430957 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baeeeabe-2781-4d6f-a851-c385d1f722f6 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:34.434563 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572157e0-a314-46e1-b5ca-b79296dd99ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 11:06:34.442685 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-572157e0-a314-46e1-b5ca-b79296dd99ac None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 29 11:06:34.442685 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:06:34.442685 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:06:34.442685 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:06:34.442685 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:06:34.442685 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:06:34.442685 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:06:34.442685 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:06:34.442685 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:06:34.442685 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:06:34.442685 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:06:34.442685 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:06:34.442685 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 11:06:34.442685 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 11:06:34.442685 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 11:06:34.442685 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 11:06:34.444159 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 11:06:34.444159 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 11:06:34.444159 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 11:06:34.444159 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 11:06:34.444159 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:06:34.444159 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:06:34.444159 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:06:34.444159 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:06:34.444159 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:06:34.444159 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:06:34.444159 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:06:34.444159 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jun 29 11:06:34.444159 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:06:34.444159 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:06:34.444159 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:06:34.444159 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:06:34.444980 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:06:34.444980 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:06:34.444980 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:06:34.444980 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:06:34.444980 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 11:06:34.444980 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 11:06:34.444980 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 11:06:34.444980 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 11:06:34.444980 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 11:06:34.444980 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 11:06:34.444980 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 29 11:06:34.444980 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jun 29 11:06:34.444980 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 231/449] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Jun 29 11:06:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:06:34.451402 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572157e0-a314-46e1-b5ca-b79296dd99ac None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:34.529167 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:34.536255 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baeeeabe-2781-4d6f-a851-c385d1f722f6 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:34.546397 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:34.553528 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572157e0-a314-46e1-b5ca-b79296dd99ac None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:34.569134 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fb2da4-4bc0-4d4c-b437-fdf8c39d2fe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:34.569713 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fb2da4-4bc0-4d4c-b437-fdf8c39d2fe2 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:34.569999 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fb2da4-4bc0-4d4c-b437-fdf8c39d2fe2 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:34.570268 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fb2da4-4bc0-4d4c-b437-fdf8c39d2fe2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:34.571091 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fb2da4-4bc0-4d4c-b437-fdf8c39d2fe2 None admin] RBAC: Adding query filter params (group.id=9451dcb3c55545be80b6512695c19e9b, role.id=e95c2ce63e114cc19a9dda6ad21e794f, scope.project.id=37897f6664b94d219c11b87aa269041a) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:06:34.571399 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fb2da4-4bc0-4d4c-b437-fdf8c39d2fe2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:34.573263 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fb2da4-4bc0-4d4c-b437-fdf8c39d2fe2 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:34.579181 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 219/450] 10.222.0.44 () {56 vars in 1376 bytes} [Wed Jun 29 11:06:34 2022] GET /identity/v3/role_assignments?group.id=9451dcb3c55545be80b6512695c19e9b&scope.project.id=37897f6664b94d219c11b87aa269041a&role.id=e95c2ce63e114cc19a9dda6ad21e794f => generated 611 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 11:06:34.588257 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f4b62b-35c6-489b-8f7f-6f41090b9990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:34.588944 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f4b62b-35c6-489b-8f7f-6f41090b9990 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:34.589223 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f4b62b-35c6-489b-8f7f-6f41090b9990 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:34.589495 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f4b62b-35c6-489b-8f7f-6f41090b9990 None admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:34.590262 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f4b62b-35c6-489b-8f7f-6f41090b9990 None admin] RBAC: Adding query filter params (name=member) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:06:34.590570 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f4b62b-35c6-489b-8f7f-6f41090b9990 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:34.592001 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f4b62b-35c6-489b-8f7f-6f41090b9990 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:34.601471 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 232/451] 10.222.0.44 () {56 vars in 1101 bytes} [Wed Jun 29 11:06:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 11:06:34.607292 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fb2da4-4bc0-4d4c-b437-fdf8c39d2fe2 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:34.712809 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fb2da4-4bc0-4d4c-b437-fdf8c39d2fe2 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:34.747245 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034d07ad-bb2d-4987-a6fe-37035021781d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:34.747929 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034d07ad-bb2d-4987-a6fe-37035021781d None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:34.748222 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034d07ad-bb2d-4987-a6fe-37035021781d None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:34.748507 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034d07ad-bb2d-4987-a6fe-37035021781d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:34.790844 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:34.815113 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034d07ad-bb2d-4987-a6fe-37035021781d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:34.816607 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034d07ad-bb2d-4987-a6fe-37035021781d None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:34.910897 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 220/452] 10.222.0.44 () {58 vars in 1282 bytes} [Wed Jun 29 11:06:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 29 11:06:34.917031 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f4b62b-35c6-489b-8f7f-6f41090b9990 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:35.012826 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f4b62b-35c6-489b-8f7f-6f41090b9990 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:35.045432 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e74d4f-1832-45c8-a7c8-0a98a3e95f3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:35.046273 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e74d4f-1832-45c8-a7c8-0a98a3e95f3c None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:35.046560 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e74d4f-1832-45c8-a7c8-0a98a3e95f3c None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:35.046836 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e74d4f-1832-45c8-a7c8-0a98a3e95f3c None admin] PATH_INFO: `/v3/projects/d36c6c9eb3234e82bb6a8537851eb336` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:35.048031 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e74d4f-1832-45c8-a7c8-0a98a3e95f3c None admin] RBAC: Authorizing `identity:get_project(project_id=d36c6c9eb3234e82bb6a8537851eb336)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:35.049475 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e74d4f-1832-45c8-a7c8-0a98a3e95f3c None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:35.051133 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 233/453] 10.222.0.44 () {56 vars in 1186 bytes} [Wed Jun 29 11:06:34 2022] GET /identity/v3/projects/d36c6c9eb3234e82bb6a8537851eb336 => generated 302 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:35.056531 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034d07ad-bb2d-4987-a6fe-37035021781d None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:35.128608 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:35.153212 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034d07ad-bb2d-4987-a6fe-37035021781d None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:35.185693 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d7c2a1-7244-4f36-93ec-a531d8c5a9e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:35.186529 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d7c2a1-7244-4f36-93ec-a531d8c5a9e1 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:35.186811 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d7c2a1-7244-4f36-93ec-a531d8c5a9e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:35.187082 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d7c2a1-7244-4f36-93ec-a531d8c5a9e1 None admin] PATH_INFO: `/v3/groups/8ec7aa0c6a3d47889a0b287a0fc483ad` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:35.188254 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d7c2a1-7244-4f36-93ec-a531d8c5a9e1 None admin] RBAC: Authorizing `identity:get_group(group_id=8ec7aa0c6a3d47889a0b287a0fc483ad)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:35.189681 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d7c2a1-7244-4f36-93ec-a531d8c5a9e1 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:35.191278 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 221/454] 10.222.0.44 () {56 vars in 1180 bytes} [Wed Jun 29 11:06:35 2022] GET /identity/v3/groups/8ec7aa0c6a3d47889a0b287a0fc483ad => generated 226 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 11:06:35.197815 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e74d4f-1832-45c8-a7c8-0a98a3e95f3c None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:35.201548 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:35.292981 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e74d4f-1832-45c8-a7c8-0a98a3e95f3c None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:35.324634 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde4b246-3f08-46d4-8d20-8786ea58c5c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:35.325268 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde4b246-3f08-46d4-8d20-8786ea58c5c5 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:35.325551 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde4b246-3f08-46d4-8d20-8786ea58c5c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:35.325826 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde4b246-3f08-46d4-8d20-8786ea58c5c5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:35.326644 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde4b246-3f08-46d4-8d20-8786ea58c5c5 None admin] RBAC: Adding query filter params (group.id=8ec7aa0c6a3d47889a0b287a0fc483ad, role.id=e1c88eebe911432cab6a07fbeb84653b, scope.project.id=d36c6c9eb3234e82bb6a8537851eb336) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:06:35.326952 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde4b246-3f08-46d4-8d20-8786ea58c5c5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:35.328393 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde4b246-3f08-46d4-8d20-8786ea58c5c5 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:35.334491 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 234/455] 10.222.0.44 () {56 vars in 1376 bytes} [Wed Jun 29 11:06:35 2022] GET /identity/v3/role_assignments?group.id=8ec7aa0c6a3d47889a0b287a0fc483ad&scope.project.id=d36c6c9eb3234e82bb6a8537851eb336&role.id=e1c88eebe911432cab6a07fbeb84653b => generated 611 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:37.974707 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fe6b85-6591-48eb-b38a-656f5400bf0b None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:37.975574 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fe6b85-6591-48eb-b38a-656f5400bf0b None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:37.976013 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fe6b85-6591-48eb-b38a-656f5400bf0b None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:37.977768 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 222/456] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:06:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:06:37.989326 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92da7766-cb2e-4a90-910a-3a262e6bdd45 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:37.989823 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92da7766-cb2e-4a90-910a-3a262e6bdd45 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:37.989909 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92da7766-cb2e-4a90-910a-3a262e6bdd45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:38.016832 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-92da7766-cb2e-4a90-910a-3a262e6bdd45 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:06:38.018043 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-92da7766-cb2e-4a90-910a-3a262e6bdd45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:06:38.109301 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:38.115754 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 235/457] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:06:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 11:06:38.155545 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fe6b85-6591-48eb-b38a-656f5400bf0b None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:38.299567 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fe6b85-6591-48eb-b38a-656f5400bf0b None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:38.345885 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62a802d-724b-4ad5-8056-8031382e6ad1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:38.346686 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62a802d-724b-4ad5-8056-8031382e6ad1 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:38.346967 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62a802d-724b-4ad5-8056-8031382e6ad1 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:38.347241 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62a802d-724b-4ad5-8056-8031382e6ad1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:38.363655 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62a802d-724b-4ad5-8056-8031382e6ad1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:38.365325 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62a802d-724b-4ad5-8056-8031382e6ad1 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:38.367257 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:38.387417 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-f62a802d-724b-4ad5-8056-8031382e6ad1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 11:06:38.387417 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:06:38.387417 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:06:38.387417 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:06:38.387417 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:06:38.387417 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:06:38.387417 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:06:38.387417 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:06:38.387417 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:06:38.387417 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:06:38.387417 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:06:38.387417 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:06:38.387417 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 11:06:38.387417 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 11:06:38.387417 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 11:06:38.396051 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 11:06:38.396051 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:06:38.396051 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:06:38.396051 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:06:38.396051 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:06:38.396051 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:06:38.396051 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:06:38.396051 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:06:38.396051 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jun 29 11:06:38.396051 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:06:38.396051 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:06:38.396051 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:06:38.396051 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:06:38.396051 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:06:38.396051 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:06:38.396051 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:06:38.396051 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:06:38.396903 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 11:06:38.396903 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 11:06:38.396903 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 11:06:38.396903 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:06:38.396903 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 223/458] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jun 29 11:06:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:06:38.397175 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92da7766-cb2e-4a90-910a-3a262e6bdd45 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:38.430518 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89d5ae9-73f0-4f0c-9208-1564511647cf None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:38.430853 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89d5ae9-73f0-4f0c-9208-1564511647cf None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:38.431082 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89d5ae9-73f0-4f0c-9208-1564511647cf None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:38.433051 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 224/459] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:06:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:06:38.445427 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1f035b-22cd-40b7-9eb6-b7faf001e569 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:38.445703 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1f035b-22cd-40b7-9eb6-b7faf001e569 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:38.445975 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1f035b-22cd-40b7-9eb6-b7faf001e569 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:38.478208 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-8e1f035b-22cd-40b7-9eb6-b7faf001e569 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:06:38.481147 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-8e1f035b-22cd-40b7-9eb6-b7faf001e569 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:06:38.579224 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92da7766-cb2e-4a90-910a-3a262e6bdd45 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:38.619205 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 225/460] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:06:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 11:06:38.623235 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb1d9b8-e8ec-4d96-8f6e-7791f4395c22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:38.625256 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb1d9b8-e8ec-4d96-8f6e-7791f4395c22 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:38.625740 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb1d9b8-e8ec-4d96-8f6e-7791f4395c22 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:38.626208 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb1d9b8-e8ec-4d96-8f6e-7791f4395c22 None admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:38.627146 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb1d9b8-e8ec-4d96-8f6e-7791f4395c22 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:06:38.627672 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb1d9b8-e8ec-4d96-8f6e-7791f4395c22 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:38.629256 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb1d9b8-e8ec-4d96-8f6e-7791f4395c22 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:38.633208 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9cb87f-04ea-48e8-9d3b-47dede69aac4 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:38.633511 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9cb87f-04ea-48e8-9d3b-47dede69aac4 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:38.633795 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9cb87f-04ea-48e8-9d3b-47dede69aac4 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:38.635402 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 226/461] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:06:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:06:38.641729 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3e8fe1-bd78-4671-aa0b-5f52a72094ea None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:38.642017 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3e8fe1-bd78-4671-aa0b-5f52a72094ea None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:38.642295 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3e8fe1-bd78-4671-aa0b-5f52a72094ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:38.643451 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 236/462] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Jun 29 11:06:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:38.651867 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb1d9b8-e8ec-4d96-8f6e-7791f4395c22 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:38.667924 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-bb3e8fe1-bd78-4671-aa0b-5f52a72094ea None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:06:38.669541 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-bb3e8fe1-bd78-4671-aa0b-5f52a72094ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:06:38.727676 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:38.744435 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 227/463] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:06:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 29 11:06:38.748286 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3e8fe1-bd78-4671-aa0b-5f52a72094ea None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:38.793211 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb1d9b8-e8ec-4d96-8f6e-7791f4395c22 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:38.830504 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:38.832765 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2651a7a4-19db-4edb-b66f-c8f3c95af2af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:38.833993 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2651a7a4-19db-4edb-b66f-c8f3c95af2af None admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:38.834280 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2651a7a4-19db-4edb-b66f-c8f3c95af2af None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:38.834572 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2651a7a4-19db-4edb-b66f-c8f3c95af2af None admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:38.835285 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2651a7a4-19db-4edb-b66f-c8f3c95af2af None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:38.836747 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2651a7a4-19db-4edb-b66f-c8f3c95af2af None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:38.875287 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3e8fe1-bd78-4671-aa0b-5f52a72094ea None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:38.909606 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd29474a-a3f8-44bc-b2a5-eff39a7c084b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:38.910397 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd29474a-a3f8-44bc-b2a5-eff39a7c084b None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:38.910683 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd29474a-a3f8-44bc-b2a5-eff39a7c084b None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:38.910995 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd29474a-a3f8-44bc-b2a5-eff39a7c084b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:38.919865 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 237/464] 10.222.0.44 () {60 vars in 1132 bytes} [Wed Jun 29 11:06:38 2022] POST /identity/v3/users => generated 246 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 11:06:38.920943 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2651a7a4-19db-4edb-b66f-c8f3c95af2af None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:38.925260 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd29474a-a3f8-44bc-b2a5-eff39a7c084b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:38.926766 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd29474a-a3f8-44bc-b2a5-eff39a7c084b None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:38.950599 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-cd29474a-a3f8-44bc-b2a5-eff39a7c084b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 11:06:38.950599 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:06:38.950599 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:06:38.950599 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:06:38.950599 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:06:38.950599 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:06:38.950599 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:06:38.950599 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:06:38.950599 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:06:38.950599 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:06:38.950599 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:06:38.950599 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:06:38.950599 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 11:06:38.950599 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 11:06:38.950599 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 11:06:38.951547 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 11:06:38.951547 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:06:38.951547 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:06:38.951547 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:06:38.951547 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:06:38.951547 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:06:38.951547 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:06:38.951547 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:06:38.951547 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jun 29 11:06:38.951547 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:06:38.951547 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:06:38.951547 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:06:38.951547 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:06:38.951547 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:06:38.951547 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:06:38.951547 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:06:38.951547 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:06:38.952431 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 11:06:38.952431 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 11:06:38.952431 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 11:06:38.952431 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:06:38.952431 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 228/465] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jun 29 11:06:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:06:38.953469 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab29458-c70c-45ca-a28e-094b8e30bf3d None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:38.959056 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab29458-c70c-45ca-a28e-094b8e30bf3d None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:38.959371 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab29458-c70c-45ca-a28e-094b8e30bf3d None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:38.964991 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 229/466] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:06:38 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:06:38.964991 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab29458-c70c-45ca-a28e-094b8e30bf3d None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:39.086151 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab29458-c70c-45ca-a28e-094b8e30bf3d None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:39.089613 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2651a7a4-19db-4edb-b66f-c8f3c95af2af None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:39.093990 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:39.105847 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:39.132880 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b190860d-c33b-4e24-8b96-922ebc317720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:39.133784 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b190860d-c33b-4e24-8b96-922ebc317720 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:39.133981 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b190860d-c33b-4e24-8b96-922ebc317720 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:39.134266 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b190860d-c33b-4e24-8b96-922ebc317720 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:39.135118 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d40e5bd-082d-4fec-8fba-31b35d68efac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:39.136822 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d40e5bd-082d-4fec-8fba-31b35d68efac None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:39.137790 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d40e5bd-082d-4fec-8fba-31b35d68efac None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:39.137790 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d40e5bd-082d-4fec-8fba-31b35d68efac None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:39.138133 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d40e5bd-082d-4fec-8fba-31b35d68efac None admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:06:39.138436 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d40e5bd-082d-4fec-8fba-31b35d68efac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:39.139988 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d40e5bd-082d-4fec-8fba-31b35d68efac None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:39.151277 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b190860d-c33b-4e24-8b96-922ebc317720 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:39.153537 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b190860d-c33b-4e24-8b96-922ebc317720 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:39.159670 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 230/467] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Jun 29 11:06:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:39.161213 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7f2baa-7970-4207-be49-b9f4386f6e55 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:39.161213 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7f2baa-7970-4207-be49-b9f4386f6e55 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:39.161991 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7f2baa-7970-4207-be49-b9f4386f6e55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:39.173821 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-b190860d-c33b-4e24-8b96-922ebc317720 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 11:06:39.173821 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:06:39.173821 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:06:39.173821 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:06:39.173821 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:06:39.173821 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:06:39.173821 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:06:39.173821 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:06:39.173821 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:06:39.173821 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:06:39.173821 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:06:39.173821 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:06:39.173821 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 11:06:39.173821 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 11:06:39.173821 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 11:06:39.174898 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 11:06:39.174898 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:06:39.174898 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:06:39.174898 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:06:39.174898 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:06:39.174898 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:06:39.174898 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:06:39.174898 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:06:39.174898 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jun 29 11:06:39.174898 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:06:39.174898 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:06:39.174898 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:06:39.174898 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:06:39.174898 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:06:39.174898 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:06:39.174898 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:06:39.174898 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:06:39.175944 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 11:06:39.175944 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 11:06:39.175944 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 11:06:39.175944 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jun 29 11:06:39.181752 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 238/468] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jun 29 11:06:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:06:39.182613 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b190860d-c33b-4e24-8b96-922ebc317720 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:39.187107 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-9a7f2baa-7970-4207-be49-b9f4386f6e55 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:06:39.188355 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-9a7f2baa-7970-4207-be49-b9f4386f6e55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:06:39.271960 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 231/469] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:06:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 11:06:39.272981 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7f2baa-7970-4207-be49-b9f4386f6e55 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:39.311671 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b190860d-c33b-4e24-8b96-922ebc317720 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:39.360080 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb6477f-88f1-4984-bc9e-cdd1dffaad7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:39.360744 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb6477f-88f1-4984-bc9e-cdd1dffaad7c None admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:39.361317 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb6477f-88f1-4984-bc9e-cdd1dffaad7c None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:39.361602 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb6477f-88f1-4984-bc9e-cdd1dffaad7c None admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:39.362403 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb6477f-88f1-4984-bc9e-cdd1dffaad7c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:39.365189 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb6477f-88f1-4984-bc9e-cdd1dffaad7c None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:39.388725 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:39.394335 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7f2baa-7970-4207-be49-b9f4386f6e55 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:39.426677 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b74c388-3c27-4784-9046-ad1309575ab2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:39.427904 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b74c388-3c27-4784-9046-ad1309575ab2 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:39.428195 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b74c388-3c27-4784-9046-ad1309575ab2 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:39.428468 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b74c388-3c27-4784-9046-ad1309575ab2 None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:39.429185 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b74c388-3c27-4784-9046-ad1309575ab2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:06:39.429490 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b74c388-3c27-4784-9046-ad1309575ab2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:39.430880 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b74c388-3c27-4784-9046-ad1309575ab2 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:39.441851 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 232/470] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Jun 29 11:06:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:39.443351 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b74c388-3c27-4784-9046-ad1309575ab2 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:39.456728 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 239/471] 10.222.0.44 () {60 vars in 1132 bytes} [Wed Jun 29 11:06:39 2022] POST /identity/v3/users => generated 248 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:06:39.458580 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbebb563-2851-4667-87f6-46b4f19891cc None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:39.459519 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbebb563-2851-4667-87f6-46b4f19891cc None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:39.460049 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbebb563-2851-4667-87f6-46b4f19891cc None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:39.466102 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 240/472] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:06:39 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:06:39.466102 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbebb563-2851-4667-87f6-46b4f19891cc None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:39.500062 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:39.566349 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b74c388-3c27-4784-9046-ad1309575ab2 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:39.584268 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbebb563-2851-4667-87f6-46b4f19891cc None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:39.606726 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4573761a-b764-4dbf-8819-ff53541678fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:39.607870 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4573761a-b764-4dbf-8819-ff53541678fd None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:39.608336 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4573761a-b764-4dbf-8819-ff53541678fd None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:39.608844 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4573761a-b764-4dbf-8819-ff53541678fd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:39.623062 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4573761a-b764-4dbf-8819-ff53541678fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:39.623976 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fdb7bc-be7c-47d4-b6c2-3d1cc0bd4542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:39.624779 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fdb7bc-be7c-47d4-b6c2-3d1cc0bd4542 None admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:39.625132 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4573761a-b764-4dbf-8819-ff53541678fd None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:39.625346 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fdb7bc-be7c-47d4-b6c2-3d1cc0bd4542 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:39.625719 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fdb7bc-be7c-47d4-b6c2-3d1cc0bd4542 None admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:39.627052 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fdb7bc-be7c-47d4-b6c2-3d1cc0bd4542 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:39.628621 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fdb7bc-be7c-47d4-b6c2-3d1cc0bd4542 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:39.646706 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-4573761a-b764-4dbf-8819-ff53541678fd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 11:06:39.646706 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:06:39.646706 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:06:39.646706 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:06:39.646706 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:06:39.646706 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:06:39.646706 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:06:39.646706 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:06:39.646706 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:06:39.646706 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:06:39.646706 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:06:39.646706 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:06:39.646706 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 11:06:39.646706 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 11:06:39.646706 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 11:06:39.647632 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 11:06:39.647632 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:06:39.647632 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:06:39.647632 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:06:39.647632 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:06:39.647632 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:06:39.647632 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:06:39.647632 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:06:39.647632 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jun 29 11:06:39.647632 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:06:39.647632 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:06:39.647632 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:06:39.647632 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:06:39.647632 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:06:39.647632 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:06:39.647632 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:06:39.647632 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:06:39.648517 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 11:06:39.648517 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 11:06:39.648517 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 11:06:39.648517 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:06:39.648517 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 233/473] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jun 29 11:06:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:06:39.650185 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017ce119-cf41-4e7b-aabb-a63e2df3b658 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:39.650185 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017ce119-cf41-4e7b-aabb-a63e2df3b658 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:39.650185 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017ce119-cf41-4e7b-aabb-a63e2df3b658 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:39.667758 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-017ce119-cf41-4e7b-aabb-a63e2df3b658 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:06:39.669004 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-017ce119-cf41-4e7b-aabb-a63e2df3b658 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:06:39.672796 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:39.743496 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 241/474] 10.222.0.44 () {60 vars in 1132 bytes} [Wed Jun 29 11:06:39 2022] POST /identity/v3/users => generated 248 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:06:39.743738 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 234/475] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:06:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 11:06:39.744562 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017ce119-cf41-4e7b-aabb-a63e2df3b658 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:39.769107 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fdb7bc-be7c-47d4-b6c2-3d1cc0bd4542 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:39.878589 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:39.902412 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fdb7bc-be7c-47d4-b6c2-3d1cc0bd4542 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:39.942773 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cfd01e-00a7-4f10-9a36-08432c8bb8a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:39.943821 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cfd01e-00a7-4f10-9a36-08432c8bb8a9 None admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:39.944369 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cfd01e-00a7-4f10-9a36-08432c8bb8a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:39.945326 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cfd01e-00a7-4f10-9a36-08432c8bb8a9 None admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:39.946453 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cfd01e-00a7-4f10-9a36-08432c8bb8a9 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:39.948328 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cfd01e-00a7-4f10-9a36-08432c8bb8a9 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:39.964411 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017ce119-cf41-4e7b-aabb-a63e2df3b658 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:40.002530 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6017e353-f538-4f7b-89e0-38f88bab7ddd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:40.003336 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6017e353-f538-4f7b-89e0-38f88bab7ddd None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:40.003713 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6017e353-f538-4f7b-89e0-38f88bab7ddd None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:40.004274 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6017e353-f538-4f7b-89e0-38f88bab7ddd None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:40.005143 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6017e353-f538-4f7b-89e0-38f88bab7ddd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:06:40.005538 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6017e353-f538-4f7b-89e0-38f88bab7ddd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:40.006995 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6017e353-f538-4f7b-89e0-38f88bab7ddd None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:40.014507 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:40.019911 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 235/476] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Jun 29 11:06:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:40.025679 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6017e353-f538-4f7b-89e0-38f88bab7ddd None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:40.038161 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.sql.core [None req-06cfd01e-00a7-4f10-9a36-08432c8bb8a9 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 29 11:06:40.038161 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 29 11:06:40.038161 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [parameters: {'id': '1228c59645f24e8582c374e832417bc8', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 29 11:06:40.038161 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80165) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 29 11:06:40.055973 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-06cfd01e-00a7-4f10-9a36-08432c8bb8a9 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 29 11:06:40.055973 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:06:40.055973 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 29 11:06:40.055973 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 29 11:06:40.055973 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 29 11:06:40.055973 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 29 11:06:40.055973 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 29 11:06:40.055973 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application result = self._query(query) Jun 29 11:06:40.055973 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 29 11:06:40.055973 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application conn.query(q) Jun 29 11:06:40.055973 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 29 11:06:40.055973 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 29 11:06:40.055973 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 29 11:06:40.055973 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application result.read() Jun 29 11:06:40.055973 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 29 11:06:40.056926 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 29 11:06:40.056926 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 29 11:06:40.056926 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 29 11:06:40.056926 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 29 11:06:40.056926 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 29 11:06:40.056926 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 29 11:06:40.056926 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 29 11:06:40.056926 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 29 11:06:40.056926 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jun 29 11:06:40.056926 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 29 11:06:40.056926 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jun 29 11:06:40.056926 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:06:40.056926 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 29 11:06:40.056926 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 29 11:06:40.056926 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jun 29 11:06:40.056926 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return ref.to_dict() Jun 29 11:06:40.056926 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 29 11:06:40.056926 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application next(self.gen) Jun 29 11:06:40.057901 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 29 11:06:40.057901 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application yield resource Jun 29 11:06:40.057901 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 29 11:06:40.057901 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application next(self.gen) Jun 29 11:06:40.057901 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 29 11:06:40.057901 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application self.session.rollback() Jun 29 11:06:40.057901 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 29 11:06:40.057901 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application self.force_reraise() Jun 29 11:06:40.057901 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 29 11:06:40.057901 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise self.value Jun 29 11:06:40.057901 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 29 11:06:40.057901 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 29 11:06:40.057901 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 29 11:06:40.057901 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application session.commit() Jun 29 11:06:40.057901 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 29 11:06:40.057901 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 29 11:06:40.057901 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 29 11:06:40.058754 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application self._prepare_impl() Jun 29 11:06:40.058754 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 29 11:06:40.058754 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application self.session.flush() Jun 29 11:06:40.058754 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 29 11:06:40.058754 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application self._flush(objects) Jun 29 11:06:40.058754 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 29 11:06:40.058754 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 29 11:06:40.058754 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 29 11:06:40.058754 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application compat.raise_( Jun 29 11:06:40.058754 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 29 11:06:40.058754 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception Jun 29 11:06:40.058754 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 29 11:06:40.058754 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application flush_context.execute() Jun 29 11:06:40.058754 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 29 11:06:40.058754 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rec.execute(self) Jun 29 11:06:40.058754 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 29 11:06:40.058754 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 29 11:06:40.059658 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 29 11:06:40.059658 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 29 11:06:40.059658 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 29 11:06:40.059658 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 29 11:06:40.059658 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 29 11:06:40.059658 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 29 11:06:40.059658 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jun 29 11:06:40.059658 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 29 11:06:40.059658 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 29 11:06:40.059658 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 29 11:06:40.059658 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 29 11:06:40.059658 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 29 11:06:40.059658 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 29 11:06:40.059658 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 29 11:06:40.059658 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 29 11:06:40.059658 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception Jun 29 11:06:40.062205 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 29 11:06:40.062205 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 29 11:06:40.062205 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 29 11:06:40.062205 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 29 11:06:40.062205 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 29 11:06:40.062205 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application result = self._query(query) Jun 29 11:06:40.062205 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 29 11:06:40.062205 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application conn.query(q) Jun 29 11:06:40.062205 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 29 11:06:40.062205 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 29 11:06:40.062205 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 29 11:06:40.062205 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application result.read() Jun 29 11:06:40.062205 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 29 11:06:40.062205 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 29 11:06:40.062205 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 29 11:06:40.062205 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 29 11:06:40.064056 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 29 11:06:40.064056 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 29 11:06:40.064056 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 29 11:06:40.064056 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 29 11:06:40.064056 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 29 11:06:40.064056 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 29 11:06:40.064056 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [parameters: {'id': '1228c59645f24e8582c374e832417bc8', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 29 11:06:40.064056 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 29 11:06:40.064056 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jun 29 11:06:40.064056 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 29 11:06:40.064056 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jun 29 11:06:40.064056 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:06:40.064056 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:06:40.064056 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:06:40.064056 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:06:40.064056 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:06:40.064865 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:06:40.064865 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:06:40.064865 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:06:40.064865 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:06:40.064865 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:06:40.064865 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:06:40.064865 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jun 29 11:06:40.064865 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jun 29 11:06:40.064865 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:06:40.064865 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:06:40.064865 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jun 29 11:06:40.064865 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jun 29 11:06:40.064865 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 29 11:06:40.064865 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 29 11:06:40.064865 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 29 11:06:40.064865 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jun 29 11:06:40.065690 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 242/477] 10.222.0.44 () {60 vars in 1131 bytes} [Wed Jun 29 11:06:39 2022] POST /identity/v3/roles => generated 144 bytes in 289 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jun 29 11:06:40.065690 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cfd01e-00a7-4f10-9a36-08432c8bb8a9 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:40.156448 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6017e353-f538-4f7b-89e0-38f88bab7ddd None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:40.187604 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cfd01e-00a7-4f10-9a36-08432c8bb8a9 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:40.195796 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81e9120-46af-4dad-b5a7-c903be3831e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:40.197188 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81e9120-46af-4dad-b5a7-c903be3831e0 None admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:40.197649 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81e9120-46af-4dad-b5a7-c903be3831e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:40.198186 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81e9120-46af-4dad-b5a7-c903be3831e0 None admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:40.199049 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81e9120-46af-4dad-b5a7-c903be3831e0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:40.200823 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81e9120-46af-4dad-b5a7-c903be3831e0 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:40.228421 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f423040a-3d09-45b9-aa74-1d7cfb3557e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:40.230089 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f423040a-3d09-45b9-aa74-1d7cfb3557e6 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:40.230089 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f423040a-3d09-45b9-aa74-1d7cfb3557e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:40.230089 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f423040a-3d09-45b9-aa74-1d7cfb3557e6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:40.241981 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f423040a-3d09-45b9-aa74-1d7cfb3557e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 11:06:40.250784 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-f423040a-3d09-45b9-aa74-1d7cfb3557e6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 29 11:06:40.250784 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:06:40.250784 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:06:40.250784 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:06:40.250784 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:06:40.250784 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:06:40.250784 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:06:40.250784 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:06:40.250784 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:06:40.250784 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:06:40.250784 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:06:40.250784 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:06:40.250784 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 11:06:40.250784 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 11:06:40.250784 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 11:06:40.250784 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 11:06:40.251689 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 11:06:40.251689 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 11:06:40.251689 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 11:06:40.251689 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 11:06:40.251689 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:06:40.251689 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:06:40.251689 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:06:40.251689 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:06:40.251689 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:06:40.251689 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:06:40.251689 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:06:40.251689 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jun 29 11:06:40.251689 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:06:40.251689 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:06:40.251689 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:06:40.251689 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:06:40.252511 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:06:40.252511 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:06:40.252511 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:06:40.252511 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:06:40.252511 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 11:06:40.252511 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 11:06:40.252511 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 11:06:40.252511 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 11:06:40.252511 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 11:06:40.252511 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 11:06:40.252511 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 29 11:06:40.252511 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jun 29 11:06:40.252511 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 243/478] 10.222.0.44 () {56 vars in 1114 bytes} [Wed Jun 29 11:06:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:06:40.257701 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f423040a-3d09-45b9-aa74-1d7cfb3557e6 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:40.261194 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:40.295482 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 236/479] 10.222.0.44 () {60 vars in 1132 bytes} [Wed Jun 29 11:06:40 2022] POST /identity/v3/users => generated 249 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 11:06:40.379097 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f423040a-3d09-45b9-aa74-1d7cfb3557e6 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:40.418442 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4c4ce4-d722-4e40-88e7-a5f1b4646580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:40.419091 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4c4ce4-d722-4e40-88e7-a5f1b4646580 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:40.419366 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4c4ce4-d722-4e40-88e7-a5f1b4646580 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:40.419819 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4c4ce4-d722-4e40-88e7-a5f1b4646580 None admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:40.420653 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4c4ce4-d722-4e40-88e7-a5f1b4646580 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:06:40.420967 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4c4ce4-d722-4e40-88e7-a5f1b4646580 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:40.422343 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4c4ce4-d722-4e40-88e7-a5f1b4646580 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:40.431790 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 244/480] 10.222.0.44 () {56 vars in 1111 bytes} [Wed Jun 29 11:06:40 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 11:06:41.294685 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc47ef2d-fd58-4eb1-88d9-9dc636c3b091 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:41.295687 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc47ef2d-fd58-4eb1-88d9-9dc636c3b091 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:41.296207 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc47ef2d-fd58-4eb1-88d9-9dc636c3b091 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:41.298178 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 237/481] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:06:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:06:41.304397 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d48ac3-7a59-4553-bb61-4090964ba72b None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:41.305348 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d48ac3-7a59-4553-bb61-4090964ba72b None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:41.305830 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d48ac3-7a59-4553-bb61-4090964ba72b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:41.323681 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-d7d48ac3-7a59-4553-bb61-4090964ba72b None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:06:41.325208 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-d7d48ac3-7a59-4553-bb61-4090964ba72b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:06:41.385066 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:41.393367 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 245/482] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:06:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 11:06:41.417210 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc47ef2d-fd58-4eb1-88d9-9dc636c3b091 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:41.493958 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:41.540399 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc47ef2d-fd58-4eb1-88d9-9dc636c3b091 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:41.573348 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f39e66-bc37-4578-851d-c355d8e304ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:41.574108 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f39e66-bc37-4578-851d-c355d8e304ab None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:41.574591 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f39e66-bc37-4578-851d-c355d8e304ab None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:41.574871 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f39e66-bc37-4578-851d-c355d8e304ab None admin] PATH_INFO: `/v3/roles/service` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:41.586174 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f39e66-bc37-4578-851d-c355d8e304ab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 11:06:41.594061 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-25f39e66-bc37-4578-851d-c355d8e304ab None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 29 11:06:41.594061 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:06:41.594061 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:06:41.594061 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:06:41.594061 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:06:41.594061 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:06:41.594061 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:06:41.594061 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:06:41.594061 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:06:41.594061 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:06:41.594061 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:06:41.594061 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:06:41.594061 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 11:06:41.594061 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 11:06:41.594061 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 11:06:41.594061 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 11:06:41.594920 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 11:06:41.594920 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 11:06:41.594920 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 11:06:41.594920 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 11:06:41.594920 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:06:41.594920 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:06:41.594920 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:06:41.594920 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:06:41.594920 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:06:41.594920 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:06:41.594920 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:06:41.594920 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jun 29 11:06:41.594920 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:06:41.594920 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:06:41.594920 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:06:41.594920 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:06:41.595695 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:06:41.595695 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:06:41.595695 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:06:41.595695 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:06:41.595695 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 11:06:41.595695 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 11:06:41.595695 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 11:06:41.595695 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 11:06:41.595695 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 11:06:41.595695 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 11:06:41.595695 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 29 11:06:41.595695 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:06:41.595695 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 238/483] 10.222.0.44 () {56 vars in 1102 bytes} [Wed Jun 29 11:06:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:06:41.602031 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d48ac3-7a59-4553-bb61-4090964ba72b None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:41.719552 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d48ac3-7a59-4553-bb61-4090964ba72b None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:41.764621 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90152da7-5fa0-4fdc-b4e0-31c4b19180d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:41.765297 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90152da7-5fa0-4fdc-b4e0-31c4b19180d8 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:41.765584 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90152da7-5fa0-4fdc-b4e0-31c4b19180d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:41.765862 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90152da7-5fa0-4fdc-b4e0-31c4b19180d8 None admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:41.766627 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90152da7-5fa0-4fdc-b4e0-31c4b19180d8 None admin] RBAC: Adding query filter params (name=service) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:06:41.766933 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90152da7-5fa0-4fdc-b4e0-31c4b19180d8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:41.768343 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90152da7-5fa0-4fdc-b4e0-31c4b19180d8 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:41.779090 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 246/484] 10.222.0.44 () {56 vars in 1103 bytes} [Wed Jun 29 11:06:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:41.784568 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f39e66-bc37-4578-851d-c355d8e304ab None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:41.947853 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f39e66-bc37-4578-851d-c355d8e304ab None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:41.968809 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:41.999582 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f80359b-251e-4dc4-b597-6353743977da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:42.000410 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f80359b-251e-4dc4-b597-6353743977da None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:42.000898 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f80359b-251e-4dc4-b597-6353743977da None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:42.001181 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f80359b-251e-4dc4-b597-6353743977da None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:42.016626 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f80359b-251e-4dc4-b597-6353743977da None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:42.018076 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f80359b-251e-4dc4-b597-6353743977da None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:42.033561 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf623f27-51a8-4cd4-af5c-7f77149d3d2c None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:42.033857 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf623f27-51a8-4cd4-af5c-7f77149d3d2c None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:42.034124 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf623f27-51a8-4cd4-af5c-7f77149d3d2c None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:42.036748 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 247/485] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:06:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 29 11:06:42.040143 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-7f80359b-251e-4dc4-b597-6353743977da None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 11:06:42.040143 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:06:42.040143 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:06:42.040143 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:06:42.040143 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:06:42.040143 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:06:42.040143 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:06:42.040143 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:06:42.040143 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:06:42.040143 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:06:42.040143 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:06:42.040143 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:06:42.040143 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 11:06:42.040143 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 11:06:42.040143 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 11:06:42.040970 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 11:06:42.040970 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:06:42.040970 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:06:42.040970 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:06:42.040970 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:06:42.040970 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:06:42.040970 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:06:42.040970 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:06:42.040970 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jun 29 11:06:42.040970 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:06:42.040970 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:06:42.040970 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:06:42.040970 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:06:42.040970 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:06:42.040970 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:06:42.040970 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:06:42.040970 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:06:42.041815 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 11:06:42.041815 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 11:06:42.041815 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 11:06:42.041815 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:06:42.041815 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 239/486] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jun 29 11:06:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:06:42.042851 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752b5fce-85e8-4188-b875-51b87d723827 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:42.043144 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752b5fce-85e8-4188-b875-51b87d723827 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:42.043420 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752b5fce-85e8-4188-b875-51b87d723827 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:42.046816 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f80359b-251e-4dc4-b597-6353743977da None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:42.074624 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-752b5fce-85e8-4188-b875-51b87d723827 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:06:42.075858 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-752b5fce-85e8-4188-b875-51b87d723827 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:06:42.164848 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:42.181797 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 248/487] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:06:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 11:06:42.218598 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752b5fce-85e8-4188-b875-51b87d723827 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:42.231721 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f80359b-251e-4dc4-b597-6353743977da None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:42.308616 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c46ec4-2fbe-423a-a7ca-6b109f131540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:42.310043 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c46ec4-2fbe-423a-a7ca-6b109f131540 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:42.310623 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c46ec4-2fbe-423a-a7ca-6b109f131540 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:42.311116 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c46ec4-2fbe-423a-a7ca-6b109f131540 None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:42.312120 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c46ec4-2fbe-423a-a7ca-6b109f131540 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:06:42.312620 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c46ec4-2fbe-423a-a7ca-6b109f131540 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:42.314173 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c46ec4-2fbe-423a-a7ca-6b109f131540 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:42.335272 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 240/488] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Jun 29 11:06:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:42.340260 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c46ec4-2fbe-423a-a7ca-6b109f131540 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:42.352660 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752b5fce-85e8-4188-b875-51b87d723827 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:42.392367 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:42.392552 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcd535e-921a-4fa6-82fd-35427e95bdbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:42.393365 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcd535e-921a-4fa6-82fd-35427e95bdbb None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:42.393580 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcd535e-921a-4fa6-82fd-35427e95bdbb None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:42.394000 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcd535e-921a-4fa6-82fd-35427e95bdbb None admin] PATH_INFO: `/v3/roles/service` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:42.406437 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcd535e-921a-4fa6-82fd-35427e95bdbb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 11:06:42.415708 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-2dcd535e-921a-4fa6-82fd-35427e95bdbb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 29 11:06:42.415708 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:06:42.415708 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:06:42.415708 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:06:42.415708 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:06:42.415708 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:06:42.415708 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:06:42.415708 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:06:42.415708 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:06:42.415708 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:06:42.415708 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:06:42.415708 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:06:42.415708 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 11:06:42.415708 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 11:06:42.415708 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 11:06:42.415708 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 11:06:42.416914 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 11:06:42.416914 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 11:06:42.416914 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 11:06:42.416914 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 11:06:42.416914 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:06:42.416914 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:06:42.416914 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:06:42.416914 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:06:42.416914 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:06:42.416914 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:06:42.416914 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:06:42.416914 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jun 29 11:06:42.416914 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:06:42.416914 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:06:42.416914 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:06:42.416914 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:06:42.417891 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:06:42.417891 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:06:42.417891 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:06:42.417891 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:06:42.417891 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 11:06:42.417891 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 11:06:42.417891 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 11:06:42.417891 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 11:06:42.417891 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 11:06:42.417891 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 11:06:42.417891 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 29 11:06:42.417891 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jun 29 11:06:42.417891 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 249/489] 10.222.0.44 () {56 vars in 1102 bytes} [Wed Jun 29 11:06:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 11:06:42.426416 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcd535e-921a-4fa6-82fd-35427e95bdbb None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:42.511465 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:42.520333 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c46ec4-2fbe-423a-a7ca-6b109f131540 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:42.557293 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89e3f6e-d148-416d-b7ec-324dc39d397f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:42.558331 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89e3f6e-d148-416d-b7ec-324dc39d397f None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:42.558754 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89e3f6e-d148-416d-b7ec-324dc39d397f None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:42.559193 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89e3f6e-d148-416d-b7ec-324dc39d397f None admin] PATH_INFO: `/v3/users/nova` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:42.567034 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89e3f6e-d148-416d-b7ec-324dc39d397f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:42.569195 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89e3f6e-d148-416d-b7ec-324dc39d397f None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:42.570386 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcd535e-921a-4fa6-82fd-35427e95bdbb None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:42.586338 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-c89e3f6e-d148-416d-b7ec-324dc39d397f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 29 11:06:42.586338 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:06:42.586338 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:06:42.586338 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:06:42.586338 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:06:42.586338 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:06:42.586338 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:06:42.586338 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:06:42.586338 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:06:42.586338 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:06:42.586338 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:06:42.586338 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:06:42.586338 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 11:06:42.586338 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 11:06:42.586338 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 11:06:42.586338 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 11:06:42.587298 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:06:42.587298 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:06:42.587298 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 11:06:42.587298 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 11:06:42.587298 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 11:06:42.587298 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 11:06:42.587298 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:06:42.587298 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:06:42.587298 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:06:42.587298 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:06:42.587298 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:06:42.587298 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jun 29 11:06:42.587298 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:06:42.587298 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:06:42.587298 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:06:42.587298 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:06:42.587298 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:06:42.587298 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:06:42.588223 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:06:42.588223 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:06:42.588223 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 11:06:42.588223 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 11:06:42.588223 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 11:06:42.588223 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 11:06:42.588223 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 11:06:42.588223 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 11:06:42.588223 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 29 11:06:42.588223 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:06:42.590903 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 241/490] 10.222.0.44 () {56 vars in 1093 bytes} [Wed Jun 29 11:06:42 2022] GET /identity/v3/users/nova => generated 82 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:06:42.592339 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aea4b8d-f407-4ac1-af3c-59ab88c6cf61 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:42.592815 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aea4b8d-f407-4ac1-af3c-59ab88c6cf61 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:42.593257 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aea4b8d-f407-4ac1-af3c-59ab88c6cf61 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:42.594952 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 242/491] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:06:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:06:42.595812 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aea4b8d-f407-4ac1-af3c-59ab88c6cf61 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:42.611546 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61eea5f-eb0c-424f-a722-0f5ad327ba97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:42.612341 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61eea5f-eb0c-424f-a722-0f5ad327ba97 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:42.612723 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61eea5f-eb0c-424f-a722-0f5ad327ba97 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:42.613043 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61eea5f-eb0c-424f-a722-0f5ad327ba97 None admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:42.613868 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61eea5f-eb0c-424f-a722-0f5ad327ba97 None admin] RBAC: Adding query filter params (name=service) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:06:42.614248 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61eea5f-eb0c-424f-a722-0f5ad327ba97 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:42.615730 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61eea5f-eb0c-424f-a722-0f5ad327ba97 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:42.624745 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 250/492] 10.222.0.44 () {56 vars in 1103 bytes} [Wed Jun 29 11:06:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:42.626069 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fc6485-2b6c-41da-a8fc-26974a98acd0 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:42.626551 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fc6485-2b6c-41da-a8fc-26974a98acd0 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:42.626769 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fc6485-2b6c-41da-a8fc-26974a98acd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:42.646591 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-99fc6485-2b6c-41da-a8fc-26974a98acd0 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:06:42.647784 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-99fc6485-2b6c-41da-a8fc-26974a98acd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:06:42.691947 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:42.710438 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aea4b8d-f407-4ac1-af3c-59ab88c6cf61 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:42.719634 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 251/493] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:06:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 11:06:42.720821 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fc6485-2b6c-41da-a8fc-26974a98acd0 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:42.747108 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832a84d4-c44e-49f5-a569-5f7216520578 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:42.748197 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832a84d4-c44e-49f5-a569-5f7216520578 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:42.748678 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832a84d4-c44e-49f5-a569-5f7216520578 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:42.749145 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832a84d4-c44e-49f5-a569-5f7216520578 None admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:42.750193 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832a84d4-c44e-49f5-a569-5f7216520578 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:06:42.750693 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832a84d4-c44e-49f5-a569-5f7216520578 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:42.752406 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832a84d4-c44e-49f5-a569-5f7216520578 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:42.762461 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:42.796727 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 243/494] 10.222.0.44 () {56 vars in 1133 bytes} [Wed Jun 29 11:06:42 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:42.797970 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832a84d4-c44e-49f5-a569-5f7216520578 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:42.834306 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fc6485-2b6c-41da-a8fc-26974a98acd0 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:42.869746 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1840171-a876-4e13-94cf-44097c8b5617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:42.870543 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1840171-a876-4e13-94cf-44097c8b5617 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:42.870890 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1840171-a876-4e13-94cf-44097c8b5617 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:42.871101 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1840171-a876-4e13-94cf-44097c8b5617 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:42.882079 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1840171-a876-4e13-94cf-44097c8b5617 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:42.883549 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1840171-a876-4e13-94cf-44097c8b5617 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:42.902154 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-d1840171-a876-4e13-94cf-44097c8b5617 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 11:06:42.902154 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:06:42.902154 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:06:42.902154 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:06:42.902154 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:06:42.902154 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:06:42.902154 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:06:42.902154 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:06:42.902154 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:06:42.902154 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:06:42.902154 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:06:42.902154 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:06:42.902154 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 11:06:42.902154 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 11:06:42.902154 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 11:06:42.903038 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 11:06:42.903038 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:06:42.903038 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:06:42.903038 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:06:42.903038 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:06:42.903038 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:06:42.903038 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:06:42.903038 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:06:42.903038 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jun 29 11:06:42.903038 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:06:42.903038 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:06:42.903038 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:06:42.903038 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:06:42.903038 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:06:42.903038 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:06:42.903038 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:06:42.903038 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:06:42.903896 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 11:06:42.903896 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 11:06:42.903896 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 11:06:42.903896 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jun 29 11:06:42.907767 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832a84d4-c44e-49f5-a569-5f7216520578 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:42.907903 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 252/495] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jun 29 11:06:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:06:42.908736 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1840171-a876-4e13-94cf-44097c8b5617 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:42.945385 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e1c79e-beac-42b3-b5f0-92ab154be8f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:42.946551 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e1c79e-beac-42b3-b5f0-92ab154be8f2 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:42.947322 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e1c79e-beac-42b3-b5f0-92ab154be8f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:42.947783 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e1c79e-beac-42b3-b5f0-92ab154be8f2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:42.959903 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e1c79e-beac-42b3-b5f0-92ab154be8f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 11:06:42.967895 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-98e1c79e-beac-42b3-b5f0-92ab154be8f2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 29 11:06:42.967895 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:06:42.967895 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:06:42.967895 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:06:42.967895 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:06:42.967895 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:06:42.967895 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:06:42.967895 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:06:42.967895 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:06:42.967895 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:06:42.967895 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:06:42.967895 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:06:42.967895 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 11:06:42.967895 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 11:06:42.967895 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 11:06:42.967895 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 11:06:42.968757 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 11:06:42.968757 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 11:06:42.968757 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 11:06:42.968757 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 11:06:42.968757 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:06:42.968757 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:06:42.968757 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:06:42.968757 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:06:42.968757 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:06:42.968757 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:06:42.968757 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:06:42.968757 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jun 29 11:06:42.968757 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:06:42.968757 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:06:42.968757 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:06:42.968757 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:06:42.969539 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:06:42.969539 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:06:42.969539 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:06:42.969539 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:06:42.969539 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 11:06:42.969539 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 11:06:42.969539 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 11:06:42.969539 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 11:06:42.969539 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 11:06:42.969539 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 11:06:42.969539 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 29 11:06:42.969539 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:06:42.971842 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 244/496] 10.222.0.44 () {56 vars in 1102 bytes} [Wed Jun 29 11:06:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:06:42.972806 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e1c79e-beac-42b3-b5f0-92ab154be8f2 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:42.976661 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:43.008334 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:43.017642 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1840171-a876-4e13-94cf-44097c8b5617 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:43.052581 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d578e922-f09f-4610-9ae7-bea4af923af3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:43.053239 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d578e922-f09f-4610-9ae7-bea4af923af3 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:43.053519 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d578e922-f09f-4610-9ae7-bea4af923af3 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:43.053790 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d578e922-f09f-4610-9ae7-bea4af923af3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:43.078035 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e1c79e-beac-42b3-b5f0-92ab154be8f2 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:43.112039 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd05690-a0bb-42e7-af92-f99920cd9779 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:43.112702 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd05690-a0bb-42e7-af92-f99920cd9779 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:43.112987 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd05690-a0bb-42e7-af92-f99920cd9779 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:43.113274 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd05690-a0bb-42e7-af92-f99920cd9779 None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:43.113992 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd05690-a0bb-42e7-af92-f99920cd9779 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:06:43.114818 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd05690-a0bb-42e7-af92-f99920cd9779 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:43.116238 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd05690-a0bb-42e7-af92-f99920cd9779 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:43.123171 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d578e922-f09f-4610-9ae7-bea4af923af3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:43.124252 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d578e922-f09f-4610-9ae7-bea4af923af3 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:43.127694 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 245/497] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Jun 29 11:06:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:43.128516 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd05690-a0bb-42e7-af92-f99920cd9779 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:43.191984 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:43.211185 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:43.221109 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 253/498] 10.222.0.44 () {58 vars in 1282 bytes} [Wed Jun 29 11:06:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 29 11:06:43.222530 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7274490-6193-42bb-b69b-43564751a5af None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:43.222849 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7274490-6193-42bb-b69b-43564751a5af None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:43.223098 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7274490-6193-42bb-b69b-43564751a5af None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:43.225075 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 254/499] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:06:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:06:43.226667 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd3aeec-65a3-4ccc-a2d2-44cfb9571a05 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:43.226956 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd3aeec-65a3-4ccc-a2d2-44cfb9571a05 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:43.227234 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd3aeec-65a3-4ccc-a2d2-44cfb9571a05 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:43.228623 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 255/500] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:06:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:06:43.229498 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd3aeec-65a3-4ccc-a2d2-44cfb9571a05 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:43.233080 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd05690-a0bb-42e7-af92-f99920cd9779 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:43.266851 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bc3d0c-15d5-4361-ab07-f3004679db4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:43.267509 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bc3d0c-15d5-4361-ab07-f3004679db4a None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:43.267872 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bc3d0c-15d5-4361-ab07-f3004679db4a None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:43.268095 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bc3d0c-15d5-4361-ab07-f3004679db4a None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:43.268833 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bc3d0c-15d5-4361-ab07-f3004679db4a None admin] RBAC: Adding query filter params (name=service) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:06:43.269138 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bc3d0c-15d5-4361-ab07-f3004679db4a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:43.270485 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bc3d0c-15d5-4361-ab07-f3004679db4a None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:43.278235 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 246/501] 10.222.0.44 () {56 vars in 1103 bytes} [Wed Jun 29 11:06:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:43.279001 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bc3d0c-15d5-4361-ab07-f3004679db4a None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:43.330118 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd3aeec-65a3-4ccc-a2d2-44cfb9571a05 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:43.365107 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c65fd1b-acd7-4230-adbe-02a3f91c1dd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:43.365898 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c65fd1b-acd7-4230-adbe-02a3f91c1dd9 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:43.366184 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c65fd1b-acd7-4230-adbe-02a3f91c1dd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:43.366459 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c65fd1b-acd7-4230-adbe-02a3f91c1dd9 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:43.373911 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c65fd1b-acd7-4230-adbe-02a3f91c1dd9 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:43.375358 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c65fd1b-acd7-4230-adbe-02a3f91c1dd9 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:43.381034 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bc3d0c-15d5-4361-ab07-f3004679db4a None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:43.390689 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-8c65fd1b-acd7-4230-adbe-02a3f91c1dd9 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 29 11:06:43.390689 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:06:43.390689 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:06:43.390689 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:06:43.390689 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:06:43.390689 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:06:43.390689 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:06:43.390689 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:06:43.390689 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:06:43.390689 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:06:43.390689 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:06:43.390689 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:06:43.390689 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 11:06:43.390689 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 11:06:43.390689 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 11:06:43.390689 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 11:06:43.391616 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:06:43.391616 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:06:43.391616 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 11:06:43.391616 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 11:06:43.391616 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 11:06:43.391616 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 11:06:43.391616 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:06:43.391616 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:06:43.391616 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:06:43.391616 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:06:43.391616 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:06:43.391616 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jun 29 11:06:43.391616 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:06:43.391616 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:06:43.391616 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:06:43.391616 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:06:43.391616 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:06:43.391616 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:06:43.392787 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:06:43.392787 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:06:43.392787 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 11:06:43.392787 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 11:06:43.392787 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 11:06:43.392787 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 11:06:43.392787 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 11:06:43.392787 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 11:06:43.392787 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 29 11:06:43.392787 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jun 29 11:06:43.392787 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 256/502] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Jun 29 11:06:43 2022] GET /identity/v3/users/glance => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:06:43.393350 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d500ca36-b237-45b9-ab0c-424b29e8e4de None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:43.393350 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d500ca36-b237-45b9-ab0c-424b29e8e4de None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:43.393593 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d500ca36-b237-45b9-ab0c-424b29e8e4de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:43.409560 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-d500ca36-b237-45b9-ab0c-424b29e8e4de None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:06:43.410360 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:43.410727 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-d500ca36-b237-45b9-ab0c-424b29e8e4de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:06:43.418778 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bb92e4-517e-4998-9a2d-991ed7d49542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:43.421956 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bb92e4-517e-4998-9a2d-991ed7d49542 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:43.421956 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bb92e4-517e-4998-9a2d-991ed7d49542 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:43.421956 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bb92e4-517e-4998-9a2d-991ed7d49542 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:43.430945 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bb92e4-517e-4998-9a2d-991ed7d49542 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:43.432512 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bb92e4-517e-4998-9a2d-991ed7d49542 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:43.449893 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-21bb92e4-517e-4998-9a2d-991ed7d49542 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 11:06:43.449893 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:06:43.449893 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:06:43.449893 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:06:43.449893 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:06:43.449893 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:06:43.449893 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:06:43.449893 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:06:43.449893 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:06:43.449893 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:06:43.449893 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:06:43.449893 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:06:43.449893 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 11:06:43.449893 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 11:06:43.449893 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 11:06:43.450882 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 11:06:43.450882 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:06:43.450882 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:06:43.450882 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:06:43.450882 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:06:43.450882 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:06:43.450882 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:06:43.450882 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:06:43.450882 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jun 29 11:06:43.450882 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:06:43.450882 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:06:43.450882 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:06:43.450882 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:06:43.450882 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:06:43.450882 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:06:43.450882 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:06:43.450882 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:06:43.451734 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 11:06:43.451734 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 11:06:43.451734 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 11:06:43.451734 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:06:43.451734 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 247/503] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jun 29 11:06:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:06:43.452289 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9041f02f-c1da-4f38-b4eb-6a2e92771f07 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:43.452548 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9041f02f-c1da-4f38-b4eb-6a2e92771f07 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:43.452822 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9041f02f-c1da-4f38-b4eb-6a2e92771f07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:43.460008 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:43.468901 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-9041f02f-c1da-4f38-b4eb-6a2e92771f07 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:06:43.470089 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-9041f02f-c1da-4f38-b4eb-6a2e92771f07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:06:43.480682 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 257/504] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:06:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 11:06:43.488657 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d500ca36-b237-45b9-ab0c-424b29e8e4de None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:43.536075 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 248/505] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:06:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 11:06:43.537356 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9041f02f-c1da-4f38-b4eb-6a2e92771f07 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:43.594072 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d500ca36-b237-45b9-ab0c-424b29e8e4de None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:43.627806 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac0b889-f6db-4213-a265-f47369bcbb64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:43.628699 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac0b889-f6db-4213-a265-f47369bcbb64 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:43.628977 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac0b889-f6db-4213-a265-f47369bcbb64 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:43.629253 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac0b889-f6db-4213-a265-f47369bcbb64 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:43.635432 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9041f02f-c1da-4f38-b4eb-6a2e92771f07 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:43.639326 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac0b889-f6db-4213-a265-f47369bcbb64 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:43.640796 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac0b889-f6db-4213-a265-f47369bcbb64 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:43.644568 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:43.657850 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-eac0b889-f6db-4213-a265-f47369bcbb64 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 11:06:43.657850 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:06:43.657850 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:06:43.657850 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:06:43.657850 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:06:43.657850 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:06:43.657850 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:06:43.657850 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:06:43.657850 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:06:43.657850 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:06:43.657850 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:06:43.657850 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:06:43.657850 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 11:06:43.657850 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 11:06:43.657850 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 11:06:43.658715 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 11:06:43.658715 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:06:43.658715 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:06:43.658715 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:06:43.658715 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:06:43.658715 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:06:43.658715 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:06:43.658715 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:06:43.658715 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jun 29 11:06:43.658715 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:06:43.658715 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:06:43.658715 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:06:43.658715 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:06:43.658715 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:06:43.658715 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:06:43.658715 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:06:43.658715 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:06:43.659553 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 11:06:43.659553 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 11:06:43.659553 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 11:06:43.659553 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jun 29 11:06:43.659553 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 258/506] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jun 29 11:06:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:06:43.659823 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac0b889-f6db-4213-a265-f47369bcbb64 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:43.669628 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83755b5-02bf-4544-b122-9f0f14c6a573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:43.670296 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83755b5-02bf-4544-b122-9f0f14c6a573 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:43.670578 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83755b5-02bf-4544-b122-9f0f14c6a573 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:43.673584 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83755b5-02bf-4544-b122-9f0f14c6a573 None admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:43.673584 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83755b5-02bf-4544-b122-9f0f14c6a573 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:06:43.673584 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83755b5-02bf-4544-b122-9f0f14c6a573 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:43.673584 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83755b5-02bf-4544-b122-9f0f14c6a573 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:43.695611 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 249/507] 10.222.0.44 () {56 vars in 1137 bytes} [Wed Jun 29 11:06:43 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:43.695611 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83755b5-02bf-4544-b122-9f0f14c6a573 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:43.695871 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:43.762229 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac0b889-f6db-4213-a265-f47369bcbb64 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:43.796524 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83755b5-02bf-4544-b122-9f0f14c6a573 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:43.796662 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b89ad15-8989-4a74-8aaf-1f7565461a11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:43.797182 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b89ad15-8989-4a74-8aaf-1f7565461a11 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:43.797467 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b89ad15-8989-4a74-8aaf-1f7565461a11 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:43.797748 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b89ad15-8989-4a74-8aaf-1f7565461a11 None admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:43.798458 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b89ad15-8989-4a74-8aaf-1f7565461a11 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:06:43.798764 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b89ad15-8989-4a74-8aaf-1f7565461a11 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:43.800170 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b89ad15-8989-4a74-8aaf-1f7565461a11 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:43.811129 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 259/508] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Jun 29 11:06:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:43.811932 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b89ad15-8989-4a74-8aaf-1f7565461a11 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:43.829794 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65867e87-2447-46a3-a8f1-71f5273b1911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:43.830578 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65867e87-2447-46a3-a8f1-71f5273b1911 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:43.830863 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65867e87-2447-46a3-a8f1-71f5273b1911 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:43.831140 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65867e87-2447-46a3-a8f1-71f5273b1911 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:43.841562 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65867e87-2447-46a3-a8f1-71f5273b1911 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:43.843010 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65867e87-2447-46a3-a8f1-71f5273b1911 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:43.866034 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-65867e87-2447-46a3-a8f1-71f5273b1911 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 11:06:43.866034 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:06:43.866034 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:06:43.866034 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:06:43.866034 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:06:43.866034 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:06:43.866034 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:06:43.866034 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:06:43.866034 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:06:43.866034 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:06:43.866034 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:06:43.866034 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:06:43.866034 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 11:06:43.866034 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 11:06:43.866034 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 11:06:43.866975 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 11:06:43.866975 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:06:43.866975 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:06:43.866975 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:06:43.866975 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:06:43.866975 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:06:43.866975 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:06:43.866975 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:06:43.866975 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jun 29 11:06:43.866975 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:06:43.866975 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:06:43.866975 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:06:43.866975 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:06:43.866975 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:06:43.866975 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:06:43.866975 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:06:43.866975 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:06:43.867919 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 11:06:43.867919 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 11:06:43.867919 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 11:06:43.867919 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:06:43.867919 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 250/509] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jun 29 11:06:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:06:43.868183 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65867e87-2447-46a3-a8f1-71f5273b1911 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:43.886727 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:43.918942 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:43.927405 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b89ad15-8989-4a74-8aaf-1f7565461a11 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:43.961171 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bb158e-f427-4cc9-9e9b-2388bd4b1ae3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:43.961965 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bb158e-f427-4cc9-9e9b-2388bd4b1ae3 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:43.962243 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bb158e-f427-4cc9-9e9b-2388bd4b1ae3 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:43.962520 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bb158e-f427-4cc9-9e9b-2388bd4b1ae3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:43.969900 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65867e87-2447-46a3-a8f1-71f5273b1911 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:43.972647 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bb158e-f427-4cc9-9e9b-2388bd4b1ae3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 11:06:43.980978 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-52bb158e-f427-4cc9-9e9b-2388bd4b1ae3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 29 11:06:43.980978 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:06:43.980978 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:06:43.980978 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:06:43.980978 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:06:43.980978 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:06:43.980978 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:06:43.980978 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:06:43.980978 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:06:43.980978 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:06:43.980978 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:06:43.980978 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:06:43.980978 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 11:06:43.980978 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 11:06:43.980978 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 11:06:43.980978 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 11:06:43.981928 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 11:06:43.981928 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 11:06:43.981928 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 11:06:43.981928 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 11:06:43.981928 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:06:43.981928 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:06:43.981928 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:06:43.981928 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:06:43.981928 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:06:43.981928 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:06:43.981928 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:06:43.981928 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jun 29 11:06:43.981928 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:06:43.981928 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:06:43.981928 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:06:43.981928 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:06:43.982729 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:06:43.982729 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:06:43.982729 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:06:43.982729 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:06:43.982729 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 11:06:43.982729 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 11:06:43.982729 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 11:06:43.982729 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 11:06:43.982729 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 11:06:43.982729 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 11:06:43.982729 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 29 11:06:43.982729 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jun 29 11:06:43.982729 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 260/510] 10.222.0.44 () {56 vars in 1102 bytes} [Wed Jun 29 11:06:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:06:43.983389 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bb158e-f427-4cc9-9e9b-2388bd4b1ae3 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:44.007720 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b91f9b0-04d7-4eb8-8550-dafd24386977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:44.008438 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b91f9b0-04d7-4eb8-8550-dafd24386977 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:44.008722 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b91f9b0-04d7-4eb8-8550-dafd24386977 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:44.008999 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b91f9b0-04d7-4eb8-8550-dafd24386977 None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:44.009704 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b91f9b0-04d7-4eb8-8550-dafd24386977 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:06:44.010006 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b91f9b0-04d7-4eb8-8550-dafd24386977 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:44.011370 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b91f9b0-04d7-4eb8-8550-dafd24386977 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:44.022960 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 251/511] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Jun 29 11:06:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:44.023744 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b91f9b0-04d7-4eb8-8550-dafd24386977 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:44.081873 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bb158e-f427-4cc9-9e9b-2388bd4b1ae3 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:44.100840 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:44.114759 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688fe140-32b7-4268-9926-9e81c20e5349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:44.115394 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688fe140-32b7-4268-9926-9e81c20e5349 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:44.115678 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688fe140-32b7-4268-9926-9e81c20e5349 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:44.117921 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688fe140-32b7-4268-9926-9e81c20e5349 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:44.121096 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b91f9b0-04d7-4eb8-8550-dafd24386977 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:44.137143 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:44.155058 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e0fe44-f686-42c1-8b94-5364a73fc99f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:44.155856 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e0fe44-f686-42c1-8b94-5364a73fc99f None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:44.157053 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e0fe44-f686-42c1-8b94-5364a73fc99f None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:44.157053 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e0fe44-f686-42c1-8b94-5364a73fc99f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:44.163209 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e0fe44-f686-42c1-8b94-5364a73fc99f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:44.164667 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e0fe44-f686-42c1-8b94-5364a73fc99f None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:44.179259 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-82e0fe44-f686-42c1-8b94-5364a73fc99f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 11:06:44.179259 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:06:44.179259 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:06:44.179259 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:06:44.179259 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:06:44.179259 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:06:44.179259 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:06:44.179259 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:06:44.179259 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:06:44.179259 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:06:44.179259 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:06:44.179259 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:06:44.179259 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 11:06:44.179259 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 11:06:44.179259 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 11:06:44.180213 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 11:06:44.180213 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:06:44.180213 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:06:44.180213 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:06:44.180213 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:06:44.180213 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:06:44.180213 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:06:44.180213 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:06:44.180213 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jun 29 11:06:44.180213 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:06:44.180213 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:06:44.180213 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:06:44.180213 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:06:44.180213 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:06:44.180213 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:06:44.180213 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:06:44.180213 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:06:44.181087 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 11:06:44.181087 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 11:06:44.181087 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 11:06:44.181087 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 11:06:44.181087 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 11:06:44.181087 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 11:06:44.181087 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 11:06:44.181087 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:06:44.181087 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 252/512] 10.222.0.44 () {56 vars in 1111 bytes} [Wed Jun 29 11:06:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:06:44.181562 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e0fe44-f686-42c1-8b94-5364a73fc99f None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:44.182968 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688fe140-32b7-4268-9926-9e81c20e5349 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:44.185410 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688fe140-32b7-4268-9926-9e81c20e5349 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:44.278431 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e0fe44-f686-42c1-8b94-5364a73fc99f None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:44.283348 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 261/513] 10.222.0.44 () {58 vars in 1282 bytes} [Wed Jun 29 11:06:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 29 11:06:44.284204 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688fe140-32b7-4268-9926-9e81c20e5349 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:44.313858 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8ded83-670b-46d6-9671-daf95bbf34ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:44.314508 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8ded83-670b-46d6-9671-daf95bbf34ac None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:44.314789 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8ded83-670b-46d6-9671-daf95bbf34ac None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:44.315060 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8ded83-670b-46d6-9671-daf95bbf34ac None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:44.315769 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8ded83-670b-46d6-9671-daf95bbf34ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:06:44.316102 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8ded83-670b-46d6-9671-daf95bbf34ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:44.318253 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8ded83-670b-46d6-9671-daf95bbf34ac None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:44.324768 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:44.329649 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 253/514] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Jun 29 11:06:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:44.330542 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8ded83-670b-46d6-9671-daf95bbf34ac None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:44.341028 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:44.381414 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688fe140-32b7-4268-9926-9e81c20e5349 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:44.413972 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3129512e-14f6-4a50-ad00-4bcb512f5b06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:44.414632 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3129512e-14f6-4a50-ad00-4bcb512f5b06 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:44.414915 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3129512e-14f6-4a50-ad00-4bcb512f5b06 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:44.415187 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3129512e-14f6-4a50-ad00-4bcb512f5b06 None admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:44.415966 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3129512e-14f6-4a50-ad00-4bcb512f5b06 None admin] RBAC: Adding query filter params (name=service) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:06:44.416271 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3129512e-14f6-4a50-ad00-4bcb512f5b06 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:44.417634 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3129512e-14f6-4a50-ad00-4bcb512f5b06 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:44.425302 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 262/515] 10.222.0.44 () {56 vars in 1103 bytes} [Wed Jun 29 11:06:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:44.426154 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3129512e-14f6-4a50-ad00-4bcb512f5b06 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:44.431405 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8ded83-670b-46d6-9671-daf95bbf34ac None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:44.508091 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541b2004-6481-489b-a10e-9b192a61bf51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:44.508878 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541b2004-6481-489b-a10e-9b192a61bf51 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:44.509161 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541b2004-6481-489b-a10e-9b192a61bf51 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:44.509435 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541b2004-6481-489b-a10e-9b192a61bf51 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:44.516888 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541b2004-6481-489b-a10e-9b192a61bf51 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:44.518343 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541b2004-6481-489b-a10e-9b192a61bf51 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:44.532195 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-541b2004-6481-489b-a10e-9b192a61bf51 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 29 11:06:44.532195 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:06:44.532195 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:06:44.532195 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:06:44.532195 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:06:44.532195 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:06:44.532195 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:06:44.532195 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:06:44.532195 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:06:44.532195 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:06:44.532195 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:06:44.532195 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:06:44.532195 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 11:06:44.532195 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 11:06:44.532195 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 11:06:44.532195 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 11:06:44.533222 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:06:44.533222 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:06:44.533222 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 11:06:44.533222 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 11:06:44.533222 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 11:06:44.533222 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 11:06:44.533222 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:06:44.533222 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:06:44.533222 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:06:44.533222 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:06:44.533222 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:06:44.533222 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jun 29 11:06:44.533222 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:06:44.533222 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:06:44.533222 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:06:44.533222 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:06:44.533222 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:06:44.533222 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:06:44.534209 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:06:44.534209 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:06:44.534209 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 11:06:44.534209 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 11:06:44.534209 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 11:06:44.534209 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 11:06:44.534209 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 11:06:44.534209 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 11:06:44.534209 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 29 11:06:44.534209 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:06:44.534209 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 254/516] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Jun 29 11:06:44 2022] GET /identity/v3/users/cinder => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:06:44.534771 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541b2004-6481-489b-a10e-9b192a61bf51 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:44.573619 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3129512e-14f6-4a50-ad00-4bcb512f5b06 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:44.586814 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:44.593461 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:44.606096 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d68de4-939d-4e47-b6be-cdad59393e39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:44.606750 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d68de4-939d-4e47-b6be-cdad59393e39 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:44.607036 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d68de4-939d-4e47-b6be-cdad59393e39 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:44.607314 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d68de4-939d-4e47-b6be-cdad59393e39 None admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:44.608130 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d68de4-939d-4e47-b6be-cdad59393e39 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:06:44.608443 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d68de4-939d-4e47-b6be-cdad59393e39 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:44.609905 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d68de4-939d-4e47-b6be-cdad59393e39 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:44.628066 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 263/517] 10.222.0.44 () {56 vars in 1148 bytes} [Wed Jun 29 11:06:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:44.628950 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d68de4-939d-4e47-b6be-cdad59393e39 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:44.634319 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541b2004-6481-489b-a10e-9b192a61bf51 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:44.667964 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c0b5c7-5df4-4e42-b8ca-f6b3d747260d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:44.668747 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c0b5c7-5df4-4e42-b8ca-f6b3d747260d None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:44.669028 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c0b5c7-5df4-4e42-b8ca-f6b3d747260d None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:44.669299 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c0b5c7-5df4-4e42-b8ca-f6b3d747260d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:44.680214 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c0b5c7-5df4-4e42-b8ca-f6b3d747260d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:44.681662 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c0b5c7-5df4-4e42-b8ca-f6b3d747260d None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:44.699239 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-26c0b5c7-5df4-4e42-b8ca-f6b3d747260d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 11:06:44.699239 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:06:44.699239 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:06:44.699239 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:06:44.699239 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:06:44.699239 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:06:44.699239 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:06:44.699239 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:06:44.699239 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:06:44.699239 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:06:44.699239 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:06:44.699239 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:06:44.699239 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 11:06:44.699239 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 11:06:44.699239 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 11:06:44.700285 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 11:06:44.700285 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:06:44.700285 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:06:44.700285 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:06:44.700285 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:06:44.700285 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:06:44.700285 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:06:44.700285 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:06:44.700285 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jun 29 11:06:44.700285 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:06:44.700285 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:06:44.700285 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:06:44.700285 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:06:44.700285 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:06:44.700285 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:06:44.700285 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:06:44.700285 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:06:44.701325 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 11:06:44.701325 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 11:06:44.701325 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 11:06:44.701325 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:06:44.701325 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 255/518] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jun 29 11:06:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:06:44.701325 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c0b5c7-5df4-4e42-b8ca-f6b3d747260d None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:44.730428 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d68de4-939d-4e47-b6be-cdad59393e39 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:44.764527 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c5e08a-3c96-4fde-af4c-9a9ade398172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:44.765202 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c5e08a-3c96-4fde-af4c-9a9ade398172 None admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:44.765490 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c5e08a-3c96-4fde-af4c-9a9ade398172 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:44.765771 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c5e08a-3c96-4fde-af4c-9a9ade398172 None admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:44.766472 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c5e08a-3c96-4fde-af4c-9a9ade398172 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:44.767954 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c5e08a-3c96-4fde-af4c-9a9ade398172 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:44.800044 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c0b5c7-5df4-4e42-b8ca-f6b3d747260d None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:44.810499 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:44.831883 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 264/519] 10.222.0.44 () {60 vars in 1132 bytes} [Wed Jun 29 11:06:44 2022] POST /identity/v3/users => generated 247 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:06:44.832919 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c5e08a-3c96-4fde-af4c-9a9ade398172 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:44.842465 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216309a8-fae7-4a70-ac6c-73dcb3b15cce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:44.843598 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216309a8-fae7-4a70-ac6c-73dcb3b15cce None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:44.844444 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216309a8-fae7-4a70-ac6c-73dcb3b15cce None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:44.845057 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216309a8-fae7-4a70-ac6c-73dcb3b15cce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:44.859759 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216309a8-fae7-4a70-ac6c-73dcb3b15cce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:44.861275 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216309a8-fae7-4a70-ac6c-73dcb3b15cce None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:44.871597 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:44.880242 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-216309a8-fae7-4a70-ac6c-73dcb3b15cce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 11:06:44.880242 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:06:44.880242 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:06:44.880242 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:06:44.880242 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:06:44.880242 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:06:44.880242 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:06:44.880242 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:06:44.880242 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:06:44.880242 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:06:44.880242 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:06:44.880242 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:06:44.880242 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 11:06:44.880242 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 11:06:44.880242 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 11:06:44.884586 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 11:06:44.884586 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:06:44.884586 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:06:44.884586 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:06:44.884586 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:06:44.884586 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:06:44.884586 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:06:44.884586 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:06:44.884586 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jun 29 11:06:44.884586 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:06:44.884586 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:06:44.884586 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:06:44.884586 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:06:44.884586 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:06:44.884586 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:06:44.884586 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:06:44.884586 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:06:44.885716 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 11:06:44.885716 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 11:06:44.885716 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 11:06:44.885716 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:06:44.885716 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 256/520] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jun 29 11:06:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:06:44.885716 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216309a8-fae7-4a70-ac6c-73dcb3b15cce None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:44.963159 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c5e08a-3c96-4fde-af4c-9a9ade398172 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:45.005439 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22dfef0-0de0-4078-beef-e1ee6a67ef63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:45.005714 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22dfef0-0de0-4078-beef-e1ee6a67ef63 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:45.006343 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22dfef0-0de0-4078-beef-e1ee6a67ef63 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:45.006343 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22dfef0-0de0-4078-beef-e1ee6a67ef63 None admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:45.006545 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216309a8-fae7-4a70-ac6c-73dcb3b15cce None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:45.007090 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22dfef0-0de0-4078-beef-e1ee6a67ef63 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:06:45.007397 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22dfef0-0de0-4078-beef-e1ee6a67ef63 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:45.008880 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22dfef0-0de0-4078-beef-e1ee6a67ef63 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:45.049584 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 265/521] 10.222.0.44 () {56 vars in 1137 bytes} [Wed Jun 29 11:06:44 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:45.049774 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423d1420-d729-496a-83c0-3f647f881e90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:45.050852 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423d1420-d729-496a-83c0-3f647f881e90 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:45.050852 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423d1420-d729-496a-83c0-3f647f881e90 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:45.051029 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22dfef0-0de0-4078-beef-e1ee6a67ef63 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:45.051128 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423d1420-d729-496a-83c0-3f647f881e90 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:45.051527 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423d1420-d729-496a-83c0-3f647f881e90 None admin] RBAC: Adding query filter params (role.id=a8212be3426741cbb1b38b1646c66405, scope.project.id=ebe5dffa4bb8440286687daefb6c98f4, user.id=796aeeff16df4739bddec1f6dee61bfe) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:06:45.052525 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423d1420-d729-496a-83c0-3f647f881e90 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:45.054056 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423d1420-d729-496a-83c0-3f647f881e90 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:45.059948 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 257/522] 10.222.0.44 () {56 vars in 1374 bytes} [Wed Jun 29 11:06:44 2022] GET /identity/v3/role_assignments?user.id=796aeeff16df4739bddec1f6dee61bfe&scope.project.id=ebe5dffa4bb8440286687daefb6c98f4&role.id=a8212be3426741cbb1b38b1646c66405 => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:45.060735 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423d1420-d729-496a-83c0-3f647f881e90 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:45.099148 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:45.176656 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:45.191454 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423d1420-d729-496a-83c0-3f647f881e90 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:45.193993 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22dfef0-0de0-4078-beef-e1ee6a67ef63 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:45.236829 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a601fc2f-ece7-4cd2-8bfc-72bbcc8c513c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:45.237645 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a601fc2f-ece7-4cd2-8bfc-72bbcc8c513c None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:45.238000 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a601fc2f-ece7-4cd2-8bfc-72bbcc8c513c None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:45.238284 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a601fc2f-ece7-4cd2-8bfc-72bbcc8c513c None admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:45.239119 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a601fc2f-ece7-4cd2-8bfc-72bbcc8c513c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:06:45.240084 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a601fc2f-ece7-4cd2-8bfc-72bbcc8c513c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:45.240209 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeef1ddb-2210-41ce-b2fe-9753eba48652 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:45.241050 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeef1ddb-2210-41ce-b2fe-9753eba48652 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:45.241481 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a601fc2f-ece7-4cd2-8bfc-72bbcc8c513c None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:45.241590 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeef1ddb-2210-41ce-b2fe-9753eba48652 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:45.241994 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeef1ddb-2210-41ce-b2fe-9753eba48652 None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:45.243129 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeef1ddb-2210-41ce-b2fe-9753eba48652 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:06:45.243585 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeef1ddb-2210-41ce-b2fe-9753eba48652 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:45.245650 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeef1ddb-2210-41ce-b2fe-9753eba48652 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:45.253946 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 266/523] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Jun 29 11:06:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:45.254811 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a601fc2f-ece7-4cd2-8bfc-72bbcc8c513c None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:45.261638 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 258/524] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Jun 29 11:06:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:45.267319 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeef1ddb-2210-41ce-b2fe-9753eba48652 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:45.363736 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a601fc2f-ece7-4cd2-8bfc-72bbcc8c513c None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:45.380739 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:45.392944 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeef1ddb-2210-41ce-b2fe-9753eba48652 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:45.407513 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8fa339-7669-4c01-b7c3-e4b802bf398c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:45.408928 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8fa339-7669-4c01-b7c3-e4b802bf398c None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:45.409369 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8fa339-7669-4c01-b7c3-e4b802bf398c None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:45.409801 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8fa339-7669-4c01-b7c3-e4b802bf398c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:45.429500 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:45.437260 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e77c22-b507-4cbd-b74a-71729b76f1eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:45.438414 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e77c22-b507-4cbd-b74a-71729b76f1eb None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:45.438706 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e77c22-b507-4cbd-b74a-71729b76f1eb None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:45.438988 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e77c22-b507-4cbd-b74a-71729b76f1eb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:45.448926 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e77c22-b507-4cbd-b74a-71729b76f1eb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:45.453027 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e77c22-b507-4cbd-b74a-71729b76f1eb None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:45.469280 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-48e77c22-b507-4cbd-b74a-71729b76f1eb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 11:06:45.469280 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:06:45.469280 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:06:45.469280 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:06:45.469280 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:06:45.469280 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:06:45.469280 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:06:45.469280 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:06:45.469280 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:06:45.469280 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:06:45.469280 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:06:45.469280 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:06:45.469280 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 11:06:45.469280 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 11:06:45.469280 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 11:06:45.470149 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 11:06:45.470149 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:06:45.470149 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:06:45.470149 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:06:45.470149 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:06:45.470149 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:06:45.470149 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:06:45.470149 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:06:45.470149 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jun 29 11:06:45.470149 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:06:45.470149 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:06:45.470149 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:06:45.470149 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:06:45.470149 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:06:45.470149 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:06:45.470149 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:06:45.470149 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:06:45.471749 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 11:06:45.471749 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 11:06:45.471749 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 11:06:45.471749 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 11:06:45.471749 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 11:06:45.471749 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 11:06:45.471749 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 11:06:45.471749 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:06:45.471749 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 259/525] 10.222.0.44 () {56 vars in 1111 bytes} [Wed Jun 29 11:06:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:06:45.471749 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e77c22-b507-4cbd-b74a-71729b76f1eb None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:45.487340 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8fa339-7669-4c01-b7c3-e4b802bf398c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:45.489184 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8fa339-7669-4c01-b7c3-e4b802bf398c None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:45.590248 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e77c22-b507-4cbd-b74a-71729b76f1eb None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:45.620321 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 267/526] 10.222.0.44 () {58 vars in 1282 bytes} [Wed Jun 29 11:06:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 29 11:06:45.621017 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8fa339-7669-4c01-b7c3-e4b802bf398c None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:45.630051 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6cb164-5a63-481f-961b-6f0c3a6e1409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:45.631132 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6cb164-5a63-481f-961b-6f0c3a6e1409 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:45.631617 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6cb164-5a63-481f-961b-6f0c3a6e1409 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:45.632325 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6cb164-5a63-481f-961b-6f0c3a6e1409 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:45.642131 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6cb164-5a63-481f-961b-6f0c3a6e1409 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:45.643794 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6cb164-5a63-481f-961b-6f0c3a6e1409 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:45.658404 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-2e6cb164-5a63-481f-961b-6f0c3a6e1409 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 29 11:06:45.658404 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:06:45.658404 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:06:45.658404 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:06:45.658404 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:06:45.658404 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:06:45.658404 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:06:45.658404 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:06:45.658404 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:06:45.658404 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:06:45.658404 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:06:45.658404 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:06:45.658404 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 11:06:45.658404 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 11:06:45.658404 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 11:06:45.658404 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 11:06:45.659278 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:06:45.659278 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:06:45.659278 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 11:06:45.659278 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 11:06:45.659278 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 11:06:45.659278 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 11:06:45.659278 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:06:45.659278 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:06:45.659278 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:06:45.659278 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:06:45.659278 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:06:45.659278 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jun 29 11:06:45.659278 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:06:45.659278 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:06:45.659278 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:06:45.659278 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:06:45.659278 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:06:45.659278 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:06:45.660201 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:06:45.660201 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:06:45.660201 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 11:06:45.660201 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 11:06:45.660201 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 11:06:45.660201 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 11:06:45.660201 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 11:06:45.660201 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 11:06:45.660201 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 29 11:06:45.660201 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:06:45.667220 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 260/527] 10.222.0.44 () {56 vars in 1102 bytes} [Wed Jun 29 11:06:45 2022] GET /identity/v3/users/neutron => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:06:45.668242 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6cb164-5a63-481f-961b-6f0c3a6e1409 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:45.676667 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:45.678296 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:45.746848 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8fa339-7669-4c01-b7c3-e4b802bf398c None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:45.782965 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8e7d30-f44e-402e-9b6a-0012eebef02f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:45.783563 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8e7d30-f44e-402e-9b6a-0012eebef02f None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:45.783867 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8e7d30-f44e-402e-9b6a-0012eebef02f None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:45.784150 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8e7d30-f44e-402e-9b6a-0012eebef02f None admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:45.784982 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8e7d30-f44e-402e-9b6a-0012eebef02f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:06:45.785292 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8e7d30-f44e-402e-9b6a-0012eebef02f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:45.786912 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8e7d30-f44e-402e-9b6a-0012eebef02f None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:45.790466 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6cb164-5a63-481f-961b-6f0c3a6e1409 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:45.799961 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 268/528] 10.222.0.44 () {56 vars in 1148 bytes} [Wed Jun 29 11:06:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:45.800846 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8e7d30-f44e-402e-9b6a-0012eebef02f None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:45.831751 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650ef868-a387-43f3-8ea7-4157fda90218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:45.832806 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650ef868-a387-43f3-8ea7-4157fda90218 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:45.833780 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650ef868-a387-43f3-8ea7-4157fda90218 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:45.834057 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650ef868-a387-43f3-8ea7-4157fda90218 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:45.845005 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650ef868-a387-43f3-8ea7-4157fda90218 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:45.846429 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650ef868-a387-43f3-8ea7-4157fda90218 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:45.864066 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-650ef868-a387-43f3-8ea7-4157fda90218 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 11:06:45.864066 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:06:45.864066 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:06:45.864066 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:06:45.864066 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:06:45.864066 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:06:45.864066 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:06:45.864066 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:06:45.864066 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:06:45.864066 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:06:45.864066 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:06:45.864066 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:06:45.864066 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 11:06:45.864066 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 11:06:45.864066 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 11:06:45.865024 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 11:06:45.865024 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:06:45.865024 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:06:45.865024 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:06:45.865024 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:06:45.865024 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:06:45.865024 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:06:45.865024 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:06:45.865024 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jun 29 11:06:45.865024 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:06:45.865024 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:06:45.865024 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:06:45.865024 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:06:45.865024 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:06:45.865024 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:06:45.865024 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:06:45.865024 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:06:45.865869 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 11:06:45.865869 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 11:06:45.865869 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 11:06:45.865869 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:06:45.870035 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 261/529] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jun 29 11:06:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:06:45.870758 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650ef868-a387-43f3-8ea7-4157fda90218 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:45.934134 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8e7d30-f44e-402e-9b6a-0012eebef02f None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:45.948732 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:45.950590 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:45.975766 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efeed4a-800a-48e8-ba19-24de811979c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:45.976487 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efeed4a-800a-48e8-ba19-24de811979c5 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:45.976771 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efeed4a-800a-48e8-ba19-24de811979c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:45.977048 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efeed4a-800a-48e8-ba19-24de811979c5 None admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:45.977895 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efeed4a-800a-48e8-ba19-24de811979c5 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:06:45.978203 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efeed4a-800a-48e8-ba19-24de811979c5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:45.979761 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efeed4a-800a-48e8-ba19-24de811979c5 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:46.002867 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 269/530] 10.222.0.44 () {56 vars in 1139 bytes} [Wed Jun 29 11:06:45 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:46.004349 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efeed4a-800a-48e8-ba19-24de811979c5 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:46.011136 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650ef868-a387-43f3-8ea7-4157fda90218 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:46.046446 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3379bf13-3546-4f51-acb2-56e9a40c1b69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:46.047146 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3379bf13-3546-4f51-acb2-56e9a40c1b69 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:46.047429 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3379bf13-3546-4f51-acb2-56e9a40c1b69 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:46.047704 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3379bf13-3546-4f51-acb2-56e9a40c1b69 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:46.048576 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3379bf13-3546-4f51-acb2-56e9a40c1b69 None admin] RBAC: Adding query filter params (role.id=a8212be3426741cbb1b38b1646c66405, scope.project.id=ebe5dffa4bb8440286687daefb6c98f4, user.id=ede3a17c7063432e8ee66462766584f8) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:06:46.048890 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3379bf13-3546-4f51-acb2-56e9a40c1b69 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:46.050301 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3379bf13-3546-4f51-acb2-56e9a40c1b69 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:46.056827 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 262/531] 10.222.0.44 () {56 vars in 1374 bytes} [Wed Jun 29 11:06:45 2022] GET /identity/v3/role_assignments?user.id=ede3a17c7063432e8ee66462766584f8&scope.project.id=ebe5dffa4bb8440286687daefb6c98f4&role.id=a8212be3426741cbb1b38b1646c66405 => generated 259 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:46.057681 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3379bf13-3546-4f51-acb2-56e9a40c1b69 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:46.139037 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efeed4a-800a-48e8-ba19-24de811979c5 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:46.165856 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3379bf13-3546-4f51-acb2-56e9a40c1b69 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:46.186054 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9273518-be09-4fb5-9b58-1ea063332f45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:46.187053 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9273518-be09-4fb5-9b58-1ea063332f45 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:46.187517 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9273518-be09-4fb5-9b58-1ea063332f45 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:46.188216 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9273518-be09-4fb5-9b58-1ea063332f45 None admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:46.188970 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9273518-be09-4fb5-9b58-1ea063332f45 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:06:46.189282 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9273518-be09-4fb5-9b58-1ea063332f45 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:46.192042 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9273518-be09-4fb5-9b58-1ea063332f45 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:46.194855 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:46.216006 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 270/532] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Jun 29 11:06:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:46.221347 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb98485e-7fb2-4082-9b94-018a109f3a67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:46.222322 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb98485e-7fb2-4082-9b94-018a109f3a67 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:46.222799 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb98485e-7fb2-4082-9b94-018a109f3a67 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:46.224326 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9273518-be09-4fb5-9b58-1ea063332f45 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:46.225056 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb98485e-7fb2-4082-9b94-018a109f3a67 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:46.278296 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:46.350580 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb98485e-7fb2-4082-9b94-018a109f3a67 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:46.352084 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb98485e-7fb2-4082-9b94-018a109f3a67 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:46.397503 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9273518-be09-4fb5-9b58-1ea063332f45 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:46.449425 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f4f2a0-14ad-43cc-9485-e7ee77d27066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:46.450221 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f4f2a0-14ad-43cc-9485-e7ee77d27066 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:46.450504 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f4f2a0-14ad-43cc-9485-e7ee77d27066 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:46.450777 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f4f2a0-14ad-43cc-9485-e7ee77d27066 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:46.464706 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f4f2a0-14ad-43cc-9485-e7ee77d27066 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:46.466379 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f4f2a0-14ad-43cc-9485-e7ee77d27066 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:46.490657 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-90f4f2a0-14ad-43cc-9485-e7ee77d27066 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 11:06:46.490657 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:06:46.490657 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:06:46.490657 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:06:46.490657 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:06:46.490657 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:06:46.490657 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:06:46.490657 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:06:46.490657 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:06:46.490657 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:06:46.490657 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:06:46.490657 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:06:46.490657 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 11:06:46.490657 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 11:06:46.490657 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 11:06:46.491551 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 11:06:46.491551 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:06:46.491551 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:06:46.491551 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:06:46.491551 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:06:46.491551 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:06:46.491551 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:06:46.491551 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:06:46.491551 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jun 29 11:06:46.491551 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:06:46.491551 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:06:46.491551 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:06:46.491551 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:06:46.491551 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:06:46.491551 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:06:46.491551 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:06:46.491551 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:06:46.492420 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 11:06:46.492420 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 11:06:46.492420 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 11:06:46.492420 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 11:06:46.492420 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 11:06:46.492420 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 11:06:46.492420 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 11:06:46.492420 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jun 29 11:06:46.493356 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 271/533] 10.222.0.44 () {56 vars in 1111 bytes} [Wed Jun 29 11:06:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 11:06:46.497417 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f4f2a0-14ad-43cc-9485-e7ee77d27066 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:46.541035 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 263/534] 10.222.0.44 () {58 vars in 1282 bytes} [Wed Jun 29 11:06:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 483 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 29 11:06:46.546695 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb98485e-7fb2-4082-9b94-018a109f3a67 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:46.574131 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:46.673130 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb98485e-7fb2-4082-9b94-018a109f3a67 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:46.674575 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:46.695889 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f4f2a0-14ad-43cc-9485-e7ee77d27066 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:46.726134 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d812f8-57a2-4d30-a196-41c6eaf3b68c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:46.727292 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d812f8-57a2-4d30-a196-41c6eaf3b68c None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:46.727767 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d812f8-57a2-4d30-a196-41c6eaf3b68c None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:46.728272 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d812f8-57a2-4d30-a196-41c6eaf3b68c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:46.731440 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da2b329-ce6e-4353-acb7-4ee273b099f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:46.732298 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da2b329-ce6e-4353-acb7-4ee273b099f6 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:46.732699 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da2b329-ce6e-4353-acb7-4ee273b099f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:46.732980 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da2b329-ce6e-4353-acb7-4ee273b099f6 None admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:46.733766 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da2b329-ce6e-4353-acb7-4ee273b099f6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:06:46.734073 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da2b329-ce6e-4353-acb7-4ee273b099f6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:46.735517 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da2b329-ce6e-4353-acb7-4ee273b099f6 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:46.741184 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d812f8-57a2-4d30-a196-41c6eaf3b68c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:46.742841 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d812f8-57a2-4d30-a196-41c6eaf3b68c None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:46.748759 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 272/535] 10.222.0.44 () {56 vars in 1148 bytes} [Wed Jun 29 11:06:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:46.754041 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da2b329-ce6e-4353-acb7-4ee273b099f6 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:46.761755 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-e1d812f8-57a2-4d30-a196-41c6eaf3b68c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 11:06:46.761755 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:06:46.761755 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:06:46.761755 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:06:46.761755 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:06:46.761755 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:06:46.761755 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:06:46.761755 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:06:46.761755 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:06:46.761755 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:06:46.761755 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:06:46.761755 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:06:46.761755 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 11:06:46.761755 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 11:06:46.761755 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 11:06:46.762684 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 11:06:46.762684 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:06:46.762684 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:06:46.762684 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:06:46.762684 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:06:46.762684 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:06:46.762684 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:06:46.762684 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:06:46.762684 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jun 29 11:06:46.762684 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:06:46.762684 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:06:46.762684 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:06:46.762684 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:06:46.762684 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:06:46.762684 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:06:46.762684 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:06:46.762684 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:06:46.763509 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 11:06:46.763509 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 11:06:46.763509 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 11:06:46.763509 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:06:46.765661 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 264/536] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jun 29 11:06:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 11:06:46.771090 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d812f8-57a2-4d30-a196-41c6eaf3b68c None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:46.902703 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:46.956668 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d812f8-57a2-4d30-a196-41c6eaf3b68c None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:46.979390 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da2b329-ce6e-4353-acb7-4ee273b099f6 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:47.019468 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61326af-c22d-42d3-8764-40154618409b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:47.020164 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61326af-c22d-42d3-8764-40154618409b None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:47.020485 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61326af-c22d-42d3-8764-40154618409b None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:47.020770 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61326af-c22d-42d3-8764-40154618409b None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:47.021478 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61326af-c22d-42d3-8764-40154618409b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:06:47.021780 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61326af-c22d-42d3-8764-40154618409b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:47.023606 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61326af-c22d-42d3-8764-40154618409b None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:47.034609 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17af2c59-5a93-4a87-a10b-3d6d81667576 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:47.035941 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17af2c59-5a93-4a87-a10b-3d6d81667576 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:47.036573 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17af2c59-5a93-4a87-a10b-3d6d81667576 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:47.037042 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17af2c59-5a93-4a87-a10b-3d6d81667576 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:47.037645 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 265/537] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Jun 29 11:06:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 11:06:47.042411 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17af2c59-5a93-4a87-a10b-3d6d81667576 None admin] RBAC: Adding query filter params (role.id=a8212be3426741cbb1b38b1646c66405, scope.project.id=ebe5dffa4bb8440286687daefb6c98f4, user.id=ff7931b75a924394b1af733febce0834) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:06:47.042579 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878fcf18-1140-4a11-af17-1ecdcf9c3b10 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:47.042579 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878fcf18-1140-4a11-af17-1ecdcf9c3b10 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:47.042579 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878fcf18-1140-4a11-af17-1ecdcf9c3b10 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:47.043313 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17af2c59-5a93-4a87-a10b-3d6d81667576 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:47.044861 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 266/538] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:06:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:06:47.045254 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17af2c59-5a93-4a87-a10b-3d6d81667576 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:47.045626 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878fcf18-1140-4a11-af17-1ecdcf9c3b10 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:47.053847 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 273/539] 10.222.0.44 () {56 vars in 1374 bytes} [Wed Jun 29 11:06:46 2022] GET /identity/v3/role_assignments?user.id=ff7931b75a924394b1af733febce0834&scope.project.id=ebe5dffa4bb8440286687daefb6c98f4&role.id=a8212be3426741cbb1b38b1646c66405 => generated 259 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:47.055418 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e49e78-a9d8-4ea4-b961-20a6c0877046 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:47.055957 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e49e78-a9d8-4ea4-b961-20a6c0877046 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:47.056466 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e49e78-a9d8-4ea4-b961-20a6c0877046 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:47.090842 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-25e49e78-a9d8-4ea4-b961-20a6c0877046 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:06:47.094683 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-25e49e78-a9d8-4ea4-b961-20a6c0877046 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:06:47.104291 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:47.223362 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 274/540] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:06:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 11:06:47.236585 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878fcf18-1140-4a11-af17-1ecdcf9c3b10 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:47.248408 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e49e78-a9d8-4ea4-b961-20a6c0877046 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:47.295050 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882f16e4-9ceb-492d-a69e-33ea7cda36f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:47.295739 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882f16e4-9ceb-492d-a69e-33ea7cda36f9 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:47.296047 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882f16e4-9ceb-492d-a69e-33ea7cda36f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:47.296325 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882f16e4-9ceb-492d-a69e-33ea7cda36f9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:47.301463 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:47.307600 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882f16e4-9ceb-492d-a69e-33ea7cda36f9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:47.309215 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882f16e4-9ceb-492d-a69e-33ea7cda36f9 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:47.330986 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-882f16e4-9ceb-492d-a69e-33ea7cda36f9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 11:06:47.330986 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:06:47.330986 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:06:47.330986 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:06:47.330986 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:06:47.330986 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:06:47.330986 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:06:47.330986 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:06:47.330986 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:06:47.330986 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:06:47.330986 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:06:47.330986 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:06:47.330986 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 11:06:47.330986 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 11:06:47.330986 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 11:06:47.331942 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 11:06:47.331942 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:06:47.331942 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:06:47.331942 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:06:47.331942 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:06:47.331942 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:06:47.331942 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:06:47.331942 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:06:47.331942 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jun 29 11:06:47.331942 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:06:47.331942 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:06:47.331942 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:06:47.331942 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:06:47.331942 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:06:47.331942 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:06:47.331942 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:06:47.331942 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:06:47.333611 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 11:06:47.333611 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 11:06:47.333611 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 11:06:47.333611 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 11:06:47.333611 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 11:06:47.333611 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 11:06:47.333611 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 11:06:47.333611 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:06:47.333611 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 267/541] 10.222.0.44 () {56 vars in 1111 bytes} [Wed Jun 29 11:06:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:06:47.341018 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882f16e4-9ceb-492d-a69e-33ea7cda36f9 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:47.360789 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e49e78-a9d8-4ea4-b961-20a6c0877046 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:47.403089 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9185d39-ed09-40bc-9231-93987bf32628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:47.404308 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9185d39-ed09-40bc-9231-93987bf32628 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:47.404917 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9185d39-ed09-40bc-9231-93987bf32628 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:47.405742 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9185d39-ed09-40bc-9231-93987bf32628 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:47.415868 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:47.420719 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9185d39-ed09-40bc-9231-93987bf32628 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 11:06:47.431105 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-e9185d39-ed09-40bc-9231-93987bf32628 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 29 11:06:47.431105 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:06:47.431105 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:06:47.431105 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:06:47.431105 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:06:47.431105 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:06:47.431105 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:06:47.431105 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:06:47.431105 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:06:47.431105 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:06:47.431105 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:06:47.431105 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:06:47.431105 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 11:06:47.431105 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 11:06:47.431105 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 11:06:47.431105 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 11:06:47.432069 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 11:06:47.432069 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 11:06:47.432069 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 11:06:47.432069 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 11:06:47.432069 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:06:47.432069 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:06:47.432069 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:06:47.432069 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:06:47.432069 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:06:47.432069 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:06:47.432069 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:06:47.432069 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jun 29 11:06:47.432069 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:06:47.432069 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:06:47.432069 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:06:47.432069 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:06:47.432868 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:06:47.432868 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:06:47.432868 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:06:47.432868 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:06:47.432868 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 11:06:47.432868 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 11:06:47.432868 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 11:06:47.432868 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 11:06:47.432868 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 11:06:47.432868 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 11:06:47.432868 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 29 11:06:47.432868 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jun 29 11:06:47.439972 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 275/542] 10.222.0.44 () {56 vars in 1102 bytes} [Wed Jun 29 11:06:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 11:06:47.441674 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511d0aab-0f24-4233-9d76-e48278eed05c None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:47.441960 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511d0aab-0f24-4233-9d76-e48278eed05c None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:47.442756 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511d0aab-0f24-4233-9d76-e48278eed05c None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:47.444458 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 276/543] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:06:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:06:47.445883 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511d0aab-0f24-4233-9d76-e48278eed05c None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:47.525203 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882f16e4-9ceb-492d-a69e-33ea7cda36f9 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:47.582773 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422c55da-0ceb-40cc-82ae-a081498de869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:47.583586 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422c55da-0ceb-40cc-82ae-a081498de869 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:47.583860 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422c55da-0ceb-40cc-82ae-a081498de869 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:47.584269 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422c55da-0ceb-40cc-82ae-a081498de869 None admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:47.585067 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422c55da-0ceb-40cc-82ae-a081498de869 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:06:47.585378 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422c55da-0ceb-40cc-82ae-a081498de869 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:47.586989 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422c55da-0ceb-40cc-82ae-a081498de869 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:47.610968 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 268/544] 10.222.0.44 () {56 vars in 1148 bytes} [Wed Jun 29 11:06:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:47.612829 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebf6ae2-00b6-4bab-aa84-4075aed55570 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:47.613122 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebf6ae2-00b6-4bab-aa84-4075aed55570 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:47.613397 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebf6ae2-00b6-4bab-aa84-4075aed55570 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:47.624572 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511d0aab-0f24-4233-9d76-e48278eed05c None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:47.634398 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-2ebf6ae2-00b6-4bab-aa84-4075aed55570 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:06:47.636257 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-2ebf6ae2-00b6-4bab-aa84-4075aed55570 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:06:47.667222 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa04382-88e6-4cf9-b049-77270c35f34b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:47.668793 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa04382-88e6-4cf9-b049-77270c35f34b None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:47.669593 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa04382-88e6-4cf9-b049-77270c35f34b None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:47.670109 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa04382-88e6-4cf9-b049-77270c35f34b None admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:47.671686 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa04382-88e6-4cf9-b049-77270c35f34b None admin] RBAC: Adding query filter params (name=service) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:06:47.672485 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa04382-88e6-4cf9-b049-77270c35f34b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:47.674099 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa04382-88e6-4cf9-b049-77270c35f34b None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:47.689671 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 277/545] 10.222.0.44 () {56 vars in 1103 bytes} [Wed Jun 29 11:06:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:47.689671 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa04382-88e6-4cf9-b049-77270c35f34b None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:47.701876 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:47.731850 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 269/546] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:06:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 11:06:47.731850 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebf6ae2-00b6-4bab-aa84-4075aed55570 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:47.781362 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:47.833465 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa04382-88e6-4cf9-b049-77270c35f34b None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:47.862764 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebf6ae2-00b6-4bab-aa84-4075aed55570 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:47.884801 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104c59ab-1f62-4ddb-a38e-12a911b57c1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:47.885461 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104c59ab-1f62-4ddb-a38e-12a911b57c1f None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:47.885745 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104c59ab-1f62-4ddb-a38e-12a911b57c1f None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:47.886020 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104c59ab-1f62-4ddb-a38e-12a911b57c1f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:47.887020 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104c59ab-1f62-4ddb-a38e-12a911b57c1f None admin] RBAC: Adding query filter params (role.id=a8212be3426741cbb1b38b1646c66405, scope.project.id=ebe5dffa4bb8440286687daefb6c98f4, user.id=b5f4bb9f989e4024a5da28dced646b92) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:06:47.887336 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104c59ab-1f62-4ddb-a38e-12a911b57c1f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:47.889114 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104c59ab-1f62-4ddb-a38e-12a911b57c1f None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:47.896906 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 278/547] 10.222.0.44 () {56 vars in 1374 bytes} [Wed Jun 29 11:06:47 2022] GET /identity/v3/role_assignments?user.id=b5f4bb9f989e4024a5da28dced646b92&scope.project.id=ebe5dffa4bb8440286687daefb6c98f4&role.id=a8212be3426741cbb1b38b1646c66405 => generated 259 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:47.900534 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104c59ab-1f62-4ddb-a38e-12a911b57c1f None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:47.927633 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a3ac4c-b1e9-46b7-a3fc-639bc486178b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:47.929599 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a3ac4c-b1e9-46b7-a3fc-639bc486178b None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:47.930093 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a3ac4c-b1e9-46b7-a3fc-639bc486178b None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:47.930829 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a3ac4c-b1e9-46b7-a3fc-639bc486178b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:47.947076 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a3ac4c-b1e9-46b7-a3fc-639bc486178b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:47.948942 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a3ac4c-b1e9-46b7-a3fc-639bc486178b None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:47.975158 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-85a3ac4c-b1e9-46b7-a3fc-639bc486178b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 11:06:47.975158 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:06:47.975158 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:06:47.975158 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:06:47.975158 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:06:47.975158 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:06:47.975158 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:06:47.975158 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:06:47.975158 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:06:47.975158 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:06:47.975158 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:06:47.975158 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:06:47.975158 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 11:06:47.975158 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 11:06:47.975158 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 11:06:47.976188 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 11:06:47.976188 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:06:47.976188 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:06:47.976188 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:06:47.976188 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:06:47.976188 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:06:47.976188 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:06:47.976188 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:06:47.976188 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jun 29 11:06:47.976188 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:06:47.976188 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:06:47.976188 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:06:47.976188 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:06:47.976188 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:06:47.976188 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:06:47.976188 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:06:47.976188 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:06:47.977023 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 11:06:47.977023 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 11:06:47.977023 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 11:06:47.977023 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:06:47.977801 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 270/548] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jun 29 11:06:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:06:47.984457 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a3ac4c-b1e9-46b7-a3fc-639bc486178b None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:48.061280 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104c59ab-1f62-4ddb-a38e-12a911b57c1f None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:48.064032 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:48.111472 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:48.131268 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f41be7f-4599-4631-a37a-1c91511f292e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:48.132101 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f41be7f-4599-4631-a37a-1c91511f292e None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:48.132468 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f41be7f-4599-4631-a37a-1c91511f292e None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:48.132747 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f41be7f-4599-4631-a37a-1c91511f292e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:48.158140 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f41be7f-4599-4631-a37a-1c91511f292e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 11:06:48.166072 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-2f41be7f-4599-4631-a37a-1c91511f292e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 29 11:06:48.166072 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:06:48.166072 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:06:48.166072 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:06:48.166072 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:06:48.166072 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:06:48.166072 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:06:48.166072 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:06:48.166072 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:06:48.166072 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:06:48.166072 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:06:48.166072 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:06:48.166072 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 11:06:48.166072 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 11:06:48.166072 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 11:06:48.166072 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 11:06:48.167153 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 11:06:48.167153 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 11:06:48.167153 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 11:06:48.167153 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 11:06:48.167153 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:06:48.167153 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:06:48.167153 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:06:48.167153 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:06:48.167153 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:06:48.167153 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:06:48.167153 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:06:48.167153 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jun 29 11:06:48.167153 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:06:48.167153 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:06:48.167153 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:06:48.167153 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:06:48.167999 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:06:48.167999 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:06:48.167999 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:06:48.167999 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:06:48.167999 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 11:06:48.167999 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 11:06:48.167999 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 11:06:48.167999 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 11:06:48.167999 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 11:06:48.167999 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 11:06:48.167999 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 29 11:06:48.167999 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jun 29 11:06:48.171933 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 279/549] 10.222.0.44 () {56 vars in 1102 bytes} [Wed Jun 29 11:06:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:06:48.173793 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f41be7f-4599-4631-a37a-1c91511f292e None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:48.229961 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a3ac4c-b1e9-46b7-a3fc-639bc486178b None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:48.301515 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d917cf6f-91b8-473b-8f8c-76b3a0560fa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:48.302723 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d917cf6f-91b8-473b-8f8c-76b3a0560fa8 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:48.303039 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d917cf6f-91b8-473b-8f8c-76b3a0560fa8 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:48.303316 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d917cf6f-91b8-473b-8f8c-76b3a0560fa8 None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:48.304075 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d917cf6f-91b8-473b-8f8c-76b3a0560fa8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:06:48.304381 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d917cf6f-91b8-473b-8f8c-76b3a0560fa8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:48.306536 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d917cf6f-91b8-473b-8f8c-76b3a0560fa8 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:48.313701 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f41be7f-4599-4631-a37a-1c91511f292e None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:48.332089 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 271/550] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Jun 29 11:06:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 348 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 11:06:48.338086 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d917cf6f-91b8-473b-8f8c-76b3a0560fa8 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:48.377369 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc3e796-04b8-48f1-926e-b880fa187432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:48.378040 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc3e796-04b8-48f1-926e-b880fa187432 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:48.378320 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc3e796-04b8-48f1-926e-b880fa187432 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:48.378593 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc3e796-04b8-48f1-926e-b880fa187432 None admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:48.379338 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc3e796-04b8-48f1-926e-b880fa187432 None admin] RBAC: Adding query filter params (name=service) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:06:48.379640 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc3e796-04b8-48f1-926e-b880fa187432 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:48.381710 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc3e796-04b8-48f1-926e-b880fa187432 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:48.399431 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 280/551] 10.222.0.44 () {56 vars in 1103 bytes} [Wed Jun 29 11:06:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 11:06:48.404728 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc3e796-04b8-48f1-926e-b880fa187432 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:48.475123 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:48.521165 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:48.531409 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d917cf6f-91b8-473b-8f8c-76b3a0560fa8 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:48.538977 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc3e796-04b8-48f1-926e-b880fa187432 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:48.570159 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06c471f-d5d5-4246-bc05-d415bbc04b65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:48.570889 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06c471f-d5d5-4246-bc05-d415bbc04b65 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:48.571167 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06c471f-d5d5-4246-bc05-d415bbc04b65 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:48.571446 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06c471f-d5d5-4246-bc05-d415bbc04b65 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:48.580847 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06c471f-d5d5-4246-bc05-d415bbc04b65 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:48.582591 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06c471f-d5d5-4246-bc05-d415bbc04b65 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:48.585104 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70afcd1-6c7a-422c-9de3-6912b912e8ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:48.586617 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70afcd1-6c7a-422c-9de3-6912b912e8ac None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:48.587096 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70afcd1-6c7a-422c-9de3-6912b912e8ac None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:48.587581 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70afcd1-6c7a-422c-9de3-6912b912e8ac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:48.597390 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-c06c471f-d5d5-4246-bc05-d415bbc04b65 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 29 11:06:48.597390 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:06:48.597390 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:06:48.597390 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:06:48.597390 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:06:48.597390 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:06:48.597390 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:06:48.597390 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:06:48.597390 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:06:48.597390 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:06:48.597390 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:06:48.597390 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:06:48.597390 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 11:06:48.597390 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 11:06:48.597390 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 11:06:48.597390 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 11:06:48.599146 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:06:48.599146 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:06:48.599146 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 11:06:48.599146 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 11:06:48.599146 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 11:06:48.599146 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 11:06:48.599146 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:06:48.599146 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:06:48.599146 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:06:48.599146 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:06:48.599146 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:06:48.599146 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jun 29 11:06:48.599146 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:06:48.599146 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:06:48.599146 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:06:48.599146 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:06:48.599146 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:06:48.599146 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:06:48.600104 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:06:48.600104 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:06:48.600104 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 11:06:48.600104 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 11:06:48.600104 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 11:06:48.600104 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 11:06:48.600104 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 11:06:48.600104 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 11:06:48.600104 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 29 11:06:48.600104 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:06:48.600104 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 272/552] 10.222.0.44 () {56 vars in 1096 bytes} [Wed Jun 29 11:06:48 2022] GET /identity/v3/users/swift => generated 83 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 11:06:48.600677 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88931932-6586-497e-9364-1040acadf211 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:48.601324 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88931932-6586-497e-9364-1040acadf211 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:48.601324 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88931932-6586-497e-9364-1040acadf211 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:48.602763 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 273/553] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:06:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:06:48.603483 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70afcd1-6c7a-422c-9de3-6912b912e8ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:48.607874 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70afcd1-6c7a-422c-9de3-6912b912e8ac None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:48.609392 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb34f56b-1e37-4550-af6d-00475623125a None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:48.609392 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb34f56b-1e37-4550-af6d-00475623125a None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:48.610306 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb34f56b-1e37-4550-af6d-00475623125a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:48.626728 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-e70afcd1-6c7a-422c-9de3-6912b912e8ac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 11:06:48.626728 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:06:48.626728 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:06:48.626728 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:06:48.626728 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:06:48.626728 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:06:48.626728 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:06:48.626728 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:06:48.626728 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:06:48.626728 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:06:48.626728 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:06:48.626728 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:06:48.626728 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 11:06:48.626728 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 11:06:48.626728 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 11:06:48.628171 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 11:06:48.628171 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:06:48.628171 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:06:48.628171 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:06:48.628171 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:06:48.628171 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:06:48.628171 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:06:48.628171 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:06:48.628171 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jun 29 11:06:48.628171 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:06:48.628171 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:06:48.628171 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:06:48.628171 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:06:48.628171 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:06:48.628171 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:06:48.628171 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:06:48.628171 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:06:48.629074 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 11:06:48.629074 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 11:06:48.629074 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 11:06:48.629074 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jun 29 11:06:48.629297 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-cb34f56b-1e37-4550-af6d-00475623125a None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:06:48.629850 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-cb34f56b-1e37-4550-af6d-00475623125a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:06:48.635460 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 281/554] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jun 29 11:06:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:06:48.637100 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70afcd1-6c7a-422c-9de3-6912b912e8ac None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:48.711856 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 274/555] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:06:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 11:06:48.711856 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb34f56b-1e37-4550-af6d-00475623125a None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:48.761784 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:48.765018 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70afcd1-6c7a-422c-9de3-6912b912e8ac None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:48.798384 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:48.801280 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d560d990-f19a-4d38-858a-5e5ace79705d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:48.802177 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d560d990-f19a-4d38-858a-5e5ace79705d None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:48.802602 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d560d990-f19a-4d38-858a-5e5ace79705d None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:48.803271 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d560d990-f19a-4d38-858a-5e5ace79705d None admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:48.804393 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d560d990-f19a-4d38-858a-5e5ace79705d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:06:48.804852 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d560d990-f19a-4d38-858a-5e5ace79705d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:48.806584 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d560d990-f19a-4d38-858a-5e5ace79705d None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:48.813799 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb34f56b-1e37-4550-af6d-00475623125a None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:48.830298 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 282/556] 10.222.0.44 () {56 vars in 1135 bytes} [Wed Jun 29 11:06:48 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:48.831390 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d560d990-f19a-4d38-858a-5e5ace79705d None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:48.851570 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d685d9d6-3881-465b-842f-224a710af3c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:48.852362 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d685d9d6-3881-465b-842f-224a710af3c2 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:48.852709 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d685d9d6-3881-465b-842f-224a710af3c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:48.853071 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d685d9d6-3881-465b-842f-224a710af3c2 None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:48.853852 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d685d9d6-3881-465b-842f-224a710af3c2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:06:48.854222 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d685d9d6-3881-465b-842f-224a710af3c2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:48.855652 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d685d9d6-3881-465b-842f-224a710af3c2 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:48.872403 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 275/557] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Jun 29 11:06:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:48.872403 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d685d9d6-3881-465b-842f-224a710af3c2 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:48.967225 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d560d990-f19a-4d38-858a-5e5ace79705d None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:49.013240 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51aa0cf-87d5-4cc4-b91f-f2a1102b5d82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:49.014423 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51aa0cf-87d5-4cc4-b91f-f2a1102b5d82 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:49.014993 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51aa0cf-87d5-4cc4-b91f-f2a1102b5d82 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:49.015468 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51aa0cf-87d5-4cc4-b91f-f2a1102b5d82 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:49.018667 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d685d9d6-3881-465b-842f-224a710af3c2 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:49.026959 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51aa0cf-87d5-4cc4-b91f-f2a1102b5d82 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 11:06:49.035886 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-a51aa0cf-87d5-4cc4-b91f-f2a1102b5d82 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 29 11:06:49.035886 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:06:49.035886 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:06:49.035886 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:06:49.035886 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:06:49.035886 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:06:49.035886 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:06:49.035886 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:06:49.035886 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:06:49.035886 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:06:49.035886 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:06:49.035886 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:06:49.035886 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 11:06:49.035886 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 11:06:49.035886 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 11:06:49.035886 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 11:06:49.037282 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 11:06:49.037282 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 11:06:49.037282 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 11:06:49.037282 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 11:06:49.037282 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:06:49.037282 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:06:49.037282 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:06:49.037282 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:06:49.037282 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:06:49.037282 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:06:49.037282 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:06:49.037282 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jun 29 11:06:49.037282 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:06:49.037282 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:06:49.037282 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:06:49.037282 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:06:49.038141 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:06:49.038141 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:06:49.038141 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:06:49.038141 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:06:49.038141 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 11:06:49.038141 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 11:06:49.038141 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 11:06:49.038141 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 11:06:49.038141 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 11:06:49.038141 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 11:06:49.038141 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 29 11:06:49.038141 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jun 29 11:06:49.045669 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 283/558] 10.222.0.44 () {56 vars in 1102 bytes} [Wed Jun 29 11:06:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:06:49.046786 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51aa0cf-87d5-4cc4-b91f-f2a1102b5d82 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:49.055289 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06195f7-a9fe-4cd0-8bbf-0c985c086bfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:49.056252 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06195f7-a9fe-4cd0-8bbf-0c985c086bfa None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:49.056757 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06195f7-a9fe-4cd0-8bbf-0c985c086bfa None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:49.057712 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06195f7-a9fe-4cd0-8bbf-0c985c086bfa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:49.072219 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:49.079512 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:49.159927 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06195f7-a9fe-4cd0-8bbf-0c985c086bfa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:49.164300 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06195f7-a9fe-4cd0-8bbf-0c985c086bfa None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:49.181895 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51aa0cf-87d5-4cc4-b91f-f2a1102b5d82 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:49.230395 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c1b197-da37-457d-af00-84a77d0c9d8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:49.231196 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c1b197-da37-457d-af00-84a77d0c9d8c None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:49.231482 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c1b197-da37-457d-af00-84a77d0c9d8c None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:49.231757 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c1b197-da37-457d-af00-84a77d0c9d8c None admin] PATH_INFO: `/v3/users/nova` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:49.242766 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c1b197-da37-457d-af00-84a77d0c9d8c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:49.244275 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c1b197-da37-457d-af00-84a77d0c9d8c None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:49.259345 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-12c1b197-da37-457d-af00-84a77d0c9d8c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 29 11:06:49.259345 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:06:49.259345 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:06:49.259345 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:06:49.259345 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:06:49.259345 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:06:49.259345 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:06:49.259345 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:06:49.259345 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:06:49.259345 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:06:49.259345 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:06:49.259345 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:06:49.259345 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 11:06:49.259345 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 11:06:49.259345 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 11:06:49.259345 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 11:06:49.260526 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:06:49.260526 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:06:49.260526 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 11:06:49.260526 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 11:06:49.260526 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 11:06:49.260526 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 11:06:49.260526 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:06:49.260526 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:06:49.260526 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:06:49.260526 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:06:49.260526 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:06:49.260526 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jun 29 11:06:49.260526 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:06:49.260526 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:06:49.260526 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:06:49.260526 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:06:49.260526 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:06:49.260526 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:06:49.261663 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:06:49.261663 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:06:49.261663 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 11:06:49.261663 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 11:06:49.261663 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 11:06:49.261663 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 11:06:49.261663 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 11:06:49.261663 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 11:06:49.261663 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 29 11:06:49.261663 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jun 29 11:06:49.261663 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 284/559] 10.222.0.44 () {56 vars in 1093 bytes} [Wed Jun 29 11:06:49 2022] GET /identity/v3/users/nova => generated 82 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:06:49.261663 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c1b197-da37-457d-af00-84a77d0c9d8c None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:49.281429 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 276/560] 10.222.0.44 () {58 vars in 1282 bytes} [Wed Jun 29 11:06:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 29 11:06:49.282885 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5cf29b-4c0b-4526-8f78-10c28a663a79 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:49.283245 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5cf29b-4c0b-4526-8f78-10c28a663a79 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:49.283590 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5cf29b-4c0b-4526-8f78-10c28a663a79 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:49.285087 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 277/561] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:06:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:06:49.285758 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5cf29b-4c0b-4526-8f78-10c28a663a79 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:49.337101 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:49.359915 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:49.381712 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c1b197-da37-457d-af00-84a77d0c9d8c None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:49.393028 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5cf29b-4c0b-4526-8f78-10c28a663a79 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:49.418325 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e850363-bd5b-4541-9c21-011dec748987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:49.419277 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e850363-bd5b-4541-9c21-011dec748987 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:49.419771 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e850363-bd5b-4541-9c21-011dec748987 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:49.420249 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e850363-bd5b-4541-9c21-011dec748987 None admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:49.421158 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e850363-bd5b-4541-9c21-011dec748987 None admin] RBAC: Adding query filter params (name=service) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:06:49.421632 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e850363-bd5b-4541-9c21-011dec748987 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:49.423337 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e850363-bd5b-4541-9c21-011dec748987 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:49.428170 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ed8b23-3cba-40b2-8758-b24cc2ee12a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:49.428833 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ed8b23-3cba-40b2-8758-b24cc2ee12a2 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:49.429116 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ed8b23-3cba-40b2-8758-b24cc2ee12a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:49.429390 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ed8b23-3cba-40b2-8758-b24cc2ee12a2 None admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:49.430238 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ed8b23-3cba-40b2-8758-b24cc2ee12a2 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:06:49.430520 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ed8b23-3cba-40b2-8758-b24cc2ee12a2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:49.432070 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ed8b23-3cba-40b2-8758-b24cc2ee12a2 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:49.433863 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 285/562] 10.222.0.44 () {56 vars in 1103 bytes} [Wed Jun 29 11:06:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:49.434828 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e850363-bd5b-4541-9c21-011dec748987 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:49.453736 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 278/563] 10.222.0.44 () {56 vars in 1133 bytes} [Wed Jun 29 11:06:49 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:49.455040 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d25a69-8db7-4cfb-a33b-fbf40fb53f9b None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:49.455328 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d25a69-8db7-4cfb-a33b-fbf40fb53f9b None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:49.455606 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d25a69-8db7-4cfb-a33b-fbf40fb53f9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:49.471079 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-65d25a69-8db7-4cfb-a33b-fbf40fb53f9b None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:06:49.472282 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-65d25a69-8db7-4cfb-a33b-fbf40fb53f9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:06:49.548241 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 279/564] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:06:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 11:06:49.548241 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d25a69-8db7-4cfb-a33b-fbf40fb53f9b None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:49.567591 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e850363-bd5b-4541-9c21-011dec748987 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:49.594361 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:49.605149 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac083859-10d2-4726-aaef-ebff366e3afe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:49.605951 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac083859-10d2-4726-aaef-ebff366e3afe None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:49.606240 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac083859-10d2-4726-aaef-ebff366e3afe None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:49.606524 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac083859-10d2-4726-aaef-ebff366e3afe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:49.616911 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:49.619076 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac083859-10d2-4726-aaef-ebff366e3afe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:49.620572 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac083859-10d2-4726-aaef-ebff366e3afe None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:49.638038 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-ac083859-10d2-4726-aaef-ebff366e3afe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 11:06:49.638038 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:06:49.638038 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:06:49.638038 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:06:49.638038 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:06:49.638038 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:06:49.638038 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:06:49.638038 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:06:49.638038 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:06:49.638038 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:06:49.638038 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:06:49.638038 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:06:49.638038 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 11:06:49.638038 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 11:06:49.638038 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 11:06:49.640024 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 11:06:49.640024 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:06:49.640024 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:06:49.640024 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:06:49.640024 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:06:49.640024 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:06:49.640024 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:06:49.640024 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:06:49.640024 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jun 29 11:06:49.640024 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:06:49.640024 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:06:49.640024 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:06:49.640024 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:06:49.640024 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:06:49.640024 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:06:49.640024 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:06:49.640024 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:06:49.641233 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 11:06:49.641233 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 11:06:49.641233 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 11:06:49.641233 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jun 29 11:06:49.641233 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 286/565] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jun 29 11:06:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:06:49.641233 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac083859-10d2-4726-aaef-ebff366e3afe None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:49.669616 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d25a69-8db7-4cfb-a33b-fbf40fb53f9b None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:49.706565 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eea69be-8603-47f9-9c5f-c5a2176ba931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:49.707440 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eea69be-8603-47f9-9c5f-c5a2176ba931 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:49.707726 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eea69be-8603-47f9-9c5f-c5a2176ba931 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:49.708577 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eea69be-8603-47f9-9c5f-c5a2176ba931 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:49.719286 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eea69be-8603-47f9-9c5f-c5a2176ba931 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:49.720735 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eea69be-8603-47f9-9c5f-c5a2176ba931 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:49.738522 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-5eea69be-8603-47f9-9c5f-c5a2176ba931 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 11:06:49.738522 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:06:49.738522 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:06:49.738522 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:06:49.738522 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:06:49.738522 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:06:49.738522 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:06:49.738522 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:06:49.738522 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:06:49.738522 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:06:49.738522 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:06:49.738522 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:06:49.738522 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 11:06:49.738522 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 11:06:49.738522 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 11:06:49.740047 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 11:06:49.740047 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:06:49.740047 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:06:49.740047 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:06:49.740047 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:06:49.740047 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:06:49.740047 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:06:49.740047 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:06:49.740047 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jun 29 11:06:49.740047 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:06:49.740047 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:06:49.740047 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:06:49.740047 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:06:49.740047 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:06:49.740047 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:06:49.740047 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:06:49.740047 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:06:49.740956 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 11:06:49.740956 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 11:06:49.740956 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 11:06:49.740956 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:06:49.740956 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 280/566] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jun 29 11:06:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:06:49.741241 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eea69be-8603-47f9-9c5f-c5a2176ba931 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:49.753056 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac083859-10d2-4726-aaef-ebff366e3afe None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:49.790172 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54d6bfb-4ecf-468b-87db-20dd16e504b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:49.790938 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54d6bfb-4ecf-468b-87db-20dd16e504b7 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:49.791227 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54d6bfb-4ecf-468b-87db-20dd16e504b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:49.791501 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54d6bfb-4ecf-468b-87db-20dd16e504b7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:49.802604 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54d6bfb-4ecf-468b-87db-20dd16e504b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:49.804081 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54d6bfb-4ecf-468b-87db-20dd16e504b7 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:49.822794 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-f54d6bfb-4ecf-468b-87db-20dd16e504b7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 11:06:49.822794 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:06:49.822794 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:06:49.822794 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:06:49.822794 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:06:49.822794 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:06:49.822794 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:06:49.822794 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:06:49.822794 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:06:49.822794 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:06:49.822794 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:06:49.822794 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:06:49.822794 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 11:06:49.822794 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 11:06:49.822794 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 11:06:49.823749 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 11:06:49.823749 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:06:49.823749 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:06:49.823749 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:06:49.823749 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:06:49.823749 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:06:49.823749 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:06:49.823749 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:06:49.823749 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jun 29 11:06:49.823749 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:06:49.823749 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:06:49.823749 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:06:49.823749 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:06:49.823749 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:06:49.823749 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:06:49.823749 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:06:49.823749 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:06:49.824694 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 11:06:49.824694 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 11:06:49.824694 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 11:06:49.824694 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jun 29 11:06:49.824694 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 287/567] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jun 29 11:06:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:06:49.824963 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54d6bfb-4ecf-468b-87db-20dd16e504b7 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:49.836885 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:49.852626 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eea69be-8603-47f9-9c5f-c5a2176ba931 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:49.876084 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:49.889110 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80aacdc-6c45-4639-bcb9-55a7a6bc59ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:49.889904 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80aacdc-6c45-4639-bcb9-55a7a6bc59ce None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:49.890189 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80aacdc-6c45-4639-bcb9-55a7a6bc59ce None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:49.890458 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80aacdc-6c45-4639-bcb9-55a7a6bc59ce None admin] PATH_INFO: `/v3/roles/service` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:49.905248 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80aacdc-6c45-4639-bcb9-55a7a6bc59ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 11:06:49.917536 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-a80aacdc-6c45-4639-bcb9-55a7a6bc59ce None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 29 11:06:49.917536 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:06:49.917536 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:06:49.917536 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:06:49.917536 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:06:49.917536 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:06:49.917536 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:06:49.917536 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:06:49.917536 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:06:49.917536 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:06:49.917536 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:06:49.917536 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:06:49.917536 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 11:06:49.917536 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 11:06:49.917536 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 11:06:49.917536 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 11:06:49.918563 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 11:06:49.918563 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 11:06:49.918563 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 11:06:49.918563 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 11:06:49.918563 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:06:49.918563 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:06:49.918563 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:06:49.918563 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:06:49.918563 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:06:49.918563 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:06:49.918563 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:06:49.918563 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jun 29 11:06:49.918563 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:06:49.918563 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:06:49.918563 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:06:49.918563 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:06:49.919961 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:06:49.919961 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:06:49.919961 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:06:49.919961 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:06:49.919961 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 11:06:49.919961 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 11:06:49.919961 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 11:06:49.919961 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 11:06:49.919961 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 11:06:49.919961 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 11:06:49.919961 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 29 11:06:49.919961 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:06:49.919961 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 281/568] 10.222.0.44 () {56 vars in 1102 bytes} [Wed Jun 29 11:06:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:06:49.919961 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80aacdc-6c45-4639-bcb9-55a7a6bc59ce None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:49.948907 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54d6bfb-4ecf-468b-87db-20dd16e504b7 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:49.983174 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357eccb6-88f6-482a-90b0-c1a96eb251e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:49.983967 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357eccb6-88f6-482a-90b0-c1a96eb251e1 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:49.984453 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357eccb6-88f6-482a-90b0-c1a96eb251e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:49.984577 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357eccb6-88f6-482a-90b0-c1a96eb251e1 None admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:49.985234 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357eccb6-88f6-482a-90b0-c1a96eb251e1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:06:49.985542 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357eccb6-88f6-482a-90b0-c1a96eb251e1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:49.986920 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357eccb6-88f6-482a-90b0-c1a96eb251e1 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:50.003575 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 288/569] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Jun 29 11:06:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:50.003575 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357eccb6-88f6-482a-90b0-c1a96eb251e1 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:50.035473 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80aacdc-6c45-4639-bcb9-55a7a6bc59ce None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:50.075250 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114b7c02-3edd-4aed-a2f7-5548a17c91c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:50.075945 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114b7c02-3edd-4aed-a2f7-5548a17c91c3 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:50.076247 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114b7c02-3edd-4aed-a2f7-5548a17c91c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:50.076520 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114b7c02-3edd-4aed-a2f7-5548a17c91c3 None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:50.077236 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114b7c02-3edd-4aed-a2f7-5548a17c91c3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:06:50.077541 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114b7c02-3edd-4aed-a2f7-5548a17c91c3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:50.078886 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114b7c02-3edd-4aed-a2f7-5548a17c91c3 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:50.090477 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 282/570] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Jun 29 11:06:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:50.091255 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114b7c02-3edd-4aed-a2f7-5548a17c91c3 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:50.099675 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:50.113000 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357eccb6-88f6-482a-90b0-c1a96eb251e1 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:50.115895 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:50.149713 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d59db5a-6c94-4945-a60b-9656bdb37840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:50.150382 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d59db5a-6c94-4945-a60b-9656bdb37840 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:50.150666 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d59db5a-6c94-4945-a60b-9656bdb37840 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:50.150941 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d59db5a-6c94-4945-a60b-9656bdb37840 None admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:50.151657 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d59db5a-6c94-4945-a60b-9656bdb37840 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:06:50.152008 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d59db5a-6c94-4945-a60b-9656bdb37840 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:50.153419 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d59db5a-6c94-4945-a60b-9656bdb37840 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:50.169325 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 289/571] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Jun 29 11:06:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:50.169325 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d59db5a-6c94-4945-a60b-9656bdb37840 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:50.189785 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114b7c02-3edd-4aed-a2f7-5548a17c91c3 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:50.224672 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4176f47d-2aac-476c-83bd-fcaa42f78494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:50.225352 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4176f47d-2aac-476c-83bd-fcaa42f78494 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:50.225639 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4176f47d-2aac-476c-83bd-fcaa42f78494 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:50.225915 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4176f47d-2aac-476c-83bd-fcaa42f78494 None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:50.226643 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4176f47d-2aac-476c-83bd-fcaa42f78494 None admin] RBAC: Adding query filter params (name=service) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:06:50.226946 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4176f47d-2aac-476c-83bd-fcaa42f78494 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:50.228338 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4176f47d-2aac-476c-83bd-fcaa42f78494 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:50.240598 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 283/572] 10.222.0.44 () {56 vars in 1103 bytes} [Wed Jun 29 11:06:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:50.240598 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c70f29-9d22-448d-9f28-0b117fe0f5a0 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:50.240598 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c70f29-9d22-448d-9f28-0b117fe0f5a0 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:50.240598 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c70f29-9d22-448d-9f28-0b117fe0f5a0 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:50.240598 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 284/573] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:06:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:06:50.240598 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c70f29-9d22-448d-9f28-0b117fe0f5a0 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:50.267049 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d59db5a-6c94-4945-a60b-9656bdb37840 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:50.299821 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c348d66-70da-46c4-979d-11cbb15a1ebe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:50.300618 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c348d66-70da-46c4-979d-11cbb15a1ebe None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:50.300906 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c348d66-70da-46c4-979d-11cbb15a1ebe None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:50.301184 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c348d66-70da-46c4-979d-11cbb15a1ebe None admin] PATH_INFO: `/v3/projects/service` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:50.307616 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c348d66-70da-46c4-979d-11cbb15a1ebe None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:50.309076 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c348d66-70da-46c4-979d-11cbb15a1ebe None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:50.309248 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:50.323798 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-6c348d66-70da-46c4-979d-11cbb15a1ebe None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 11:06:50.323798 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:06:50.323798 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:06:50.323798 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:06:50.323798 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:06:50.323798 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:06:50.323798 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:06:50.323798 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:06:50.323798 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:06:50.323798 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:06:50.323798 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:06:50.323798 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:06:50.323798 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 11:06:50.323798 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 11:06:50.323798 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 11:06:50.324745 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 11:06:50.324745 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:06:50.324745 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:06:50.324745 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:06:50.324745 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:06:50.324745 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:06:50.324745 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:06:50.324745 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:06:50.324745 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jun 29 11:06:50.324745 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:06:50.324745 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:06:50.324745 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:06:50.324745 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:06:50.324745 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:06:50.324745 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:06:50.324745 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:06:50.324745 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:06:50.325626 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 11:06:50.325626 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 11:06:50.325626 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 11:06:50.325626 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 11:06:50.325626 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 11:06:50.325626 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 11:06:50.325626 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 11:06:50.325626 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jun 29 11:06:50.325626 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 290/574] 10.222.0.44 () {56 vars in 1111 bytes} [Wed Jun 29 11:06:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:06:50.326098 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c348d66-70da-46c4-979d-11cbb15a1ebe None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:50.350664 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c70f29-9d22-448d-9f28-0b117fe0f5a0 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:50.350854 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:50.384559 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf2e76d-7272-4101-8a9c-cea90e9a18cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:50.385395 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf2e76d-7272-4101-8a9c-cea90e9a18cd None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:50.385682 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf2e76d-7272-4101-8a9c-cea90e9a18cd None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:50.385960 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf2e76d-7272-4101-8a9c-cea90e9a18cd None admin] PATH_INFO: `/v3/users/glance` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:50.393431 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf2e76d-7272-4101-8a9c-cea90e9a18cd None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:50.394877 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf2e76d-7272-4101-8a9c-cea90e9a18cd None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:50.408890 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-eaf2e76d-7272-4101-8a9c-cea90e9a18cd None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 29 11:06:50.408890 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:06:50.408890 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:06:50.408890 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:06:50.408890 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:06:50.408890 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:06:50.408890 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:06:50.408890 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:06:50.408890 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:06:50.408890 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:06:50.408890 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:06:50.408890 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:06:50.408890 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 11:06:50.408890 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 11:06:50.408890 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 11:06:50.408890 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 11:06:50.409995 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:06:50.409995 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:06:50.409995 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 11:06:50.409995 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 11:06:50.409995 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 11:06:50.409995 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 11:06:50.409995 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:06:50.409995 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:06:50.409995 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:06:50.409995 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:06:50.409995 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:06:50.409995 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jun 29 11:06:50.409995 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:06:50.409995 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:06:50.409995 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:06:50.409995 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:06:50.409995 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:06:50.409995 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:06:50.411477 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:06:50.411477 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:06:50.411477 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 11:06:50.411477 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 11:06:50.411477 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 11:06:50.411477 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 11:06:50.411477 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 11:06:50.411477 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 11:06:50.411477 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 29 11:06:50.411477 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:06:50.411477 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 285/575] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Jun 29 11:06:50 2022] GET /identity/v3/users/glance => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:06:50.411477 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf2e76d-7272-4101-8a9c-cea90e9a18cd None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:50.427677 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c348d66-70da-46c4-979d-11cbb15a1ebe None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:50.461837 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e3fde9-bfb4-496a-97f8-e0ada52de7db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:50.462561 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e3fde9-bfb4-496a-97f8-e0ada52de7db None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:50.462848 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e3fde9-bfb4-496a-97f8-e0ada52de7db None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:50.463126 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e3fde9-bfb4-496a-97f8-e0ada52de7db None admin] PATH_INFO: `/v3/projects/service` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:50.469710 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e3fde9-bfb4-496a-97f8-e0ada52de7db None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:50.471142 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e3fde9-bfb4-496a-97f8-e0ada52de7db None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:50.485168 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-86e3fde9-bfb4-496a-97f8-e0ada52de7db None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 11:06:50.485168 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:06:50.485168 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:06:50.485168 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:06:50.485168 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:06:50.485168 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:06:50.485168 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:06:50.485168 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:06:50.485168 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:06:50.485168 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:06:50.485168 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:06:50.485168 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:06:50.485168 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 11:06:50.485168 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 11:06:50.485168 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 11:06:50.486040 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 11:06:50.486040 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:06:50.486040 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:06:50.486040 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:06:50.486040 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:06:50.486040 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:06:50.486040 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:06:50.486040 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:06:50.486040 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jun 29 11:06:50.486040 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:06:50.486040 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:06:50.486040 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:06:50.486040 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:06:50.486040 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:06:50.486040 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:06:50.486040 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:06:50.486040 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:06:50.486956 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 11:06:50.486956 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 11:06:50.486956 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 11:06:50.486956 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 11:06:50.486956 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 11:06:50.486956 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 11:06:50.486956 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 11:06:50.486956 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jun 29 11:06:50.486956 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 291/576] 10.222.0.44 () {56 vars in 1111 bytes} [Wed Jun 29 11:06:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:06:50.487622 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77d5b7c-27b7-4ab3-bd7d-03a783347689 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:50.487791 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77d5b7c-27b7-4ab3-bd7d-03a783347689 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:50.488066 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77d5b7c-27b7-4ab3-bd7d-03a783347689 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:50.504035 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-c77d5b7c-27b7-4ab3-bd7d-03a783347689 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:06:50.505295 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-c77d5b7c-27b7-4ab3-bd7d-03a783347689 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:06:50.515779 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf2e76d-7272-4101-8a9c-cea90e9a18cd None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:50.545559 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:50.553605 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99a302a-d0db-477e-8b0c-8090ec83580e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:50.566639 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99a302a-d0db-477e-8b0c-8090ec83580e None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:50.566639 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99a302a-d0db-477e-8b0c-8090ec83580e None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:50.566639 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99a302a-d0db-477e-8b0c-8090ec83580e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:50.569936 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 292/577] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:06:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 11:06:50.570773 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77d5b7c-27b7-4ab3-bd7d-03a783347689 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:50.579942 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99a302a-d0db-477e-8b0c-8090ec83580e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:50.581403 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99a302a-d0db-477e-8b0c-8090ec83580e None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:50.587078 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:50.602087 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-c99a302a-d0db-477e-8b0c-8090ec83580e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 11:06:50.602087 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:06:50.602087 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:06:50.602087 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:06:50.602087 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:06:50.602087 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:06:50.602087 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:06:50.602087 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:06:50.602087 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:06:50.602087 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:06:50.602087 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:06:50.602087 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:06:50.602087 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 11:06:50.602087 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 11:06:50.602087 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 11:06:50.603223 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 11:06:50.603223 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:06:50.603223 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:06:50.603223 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:06:50.603223 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:06:50.603223 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:06:50.603223 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:06:50.603223 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:06:50.603223 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jun 29 11:06:50.603223 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:06:50.603223 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:06:50.603223 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:06:50.603223 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:06:50.603223 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:06:50.603223 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:06:50.603223 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:06:50.603223 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:06:50.604199 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 11:06:50.604199 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 11:06:50.604199 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 11:06:50.604199 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:06:50.604199 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 286/578] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jun 29 11:06:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:06:50.604199 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99a302a-d0db-477e-8b0c-8090ec83580e None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:50.676280 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77d5b7c-27b7-4ab3-bd7d-03a783347689 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:50.702933 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99a302a-d0db-477e-8b0c-8090ec83580e None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:50.710234 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e50c39-8b6e-4a31-9604-8854609cd20f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:50.710893 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e50c39-8b6e-4a31-9604-8854609cd20f None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:50.711177 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e50c39-8b6e-4a31-9604-8854609cd20f None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:50.711451 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e50c39-8b6e-4a31-9604-8854609cd20f None admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:50.712316 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e50c39-8b6e-4a31-9604-8854609cd20f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:06:50.712571 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e50c39-8b6e-4a31-9604-8854609cd20f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:50.713993 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e50c39-8b6e-4a31-9604-8854609cd20f None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:50.726136 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 293/579] 10.222.0.44 () {56 vars in 1148 bytes} [Wed Jun 29 11:06:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:50.726928 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e50c39-8b6e-4a31-9604-8854609cd20f None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:50.741803 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d2dab6-d5a9-4e03-bbc1-05e49d9657a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:50.742471 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d2dab6-d5a9-4e03-bbc1-05e49d9657a0 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:50.742756 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d2dab6-d5a9-4e03-bbc1-05e49d9657a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:50.743033 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d2dab6-d5a9-4e03-bbc1-05e49d9657a0 None admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:50.743924 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d2dab6-d5a9-4e03-bbc1-05e49d9657a0 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:06:50.744237 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d2dab6-d5a9-4e03-bbc1-05e49d9657a0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:50.745638 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d2dab6-d5a9-4e03-bbc1-05e49d9657a0 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:50.762620 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 287/580] 10.222.0.44 () {56 vars in 1137 bytes} [Wed Jun 29 11:06:50 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:50.763412 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d2dab6-d5a9-4e03-bbc1-05e49d9657a0 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:50.802869 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:50.803319 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:50.827372 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e50c39-8b6e-4a31-9604-8854609cd20f None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:50.861928 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7aba919-fff0-4284-a9da-045741585b84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:50.862640 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d2dab6-d5a9-4e03-bbc1-05e49d9657a0 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:50.862744 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aba919-fff0-4284-a9da-045741585b84 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:50.862953 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aba919-fff0-4284-a9da-045741585b84 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:50.863227 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aba919-fff0-4284-a9da-045741585b84 None admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:50.864065 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7aba919-fff0-4284-a9da-045741585b84 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:06:50.864374 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7aba919-fff0-4284-a9da-045741585b84 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:50.865810 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7aba919-fff0-4284-a9da-045741585b84 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:50.877501 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 294/581] 10.222.0.44 () {56 vars in 1148 bytes} [Wed Jun 29 11:06:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:50.878284 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7aba919-fff0-4284-a9da-045741585b84 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:50.900986 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c4cf33-b7dc-4f43-b49b-86e3101ad224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:50.901777 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c4cf33-b7dc-4f43-b49b-86e3101ad224 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:50.902056 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c4cf33-b7dc-4f43-b49b-86e3101ad224 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:50.902334 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c4cf33-b7dc-4f43-b49b-86e3101ad224 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:50.914576 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c4cf33-b7dc-4f43-b49b-86e3101ad224 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 11:06:50.921463 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-a8c4cf33-b7dc-4f43-b49b-86e3101ad224 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 29 11:06:50.921463 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:06:50.921463 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:06:50.921463 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:06:50.921463 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:06:50.921463 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:06:50.921463 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:06:50.921463 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:06:50.921463 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:06:50.921463 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:06:50.921463 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:06:50.921463 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:06:50.921463 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 11:06:50.921463 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 11:06:50.921463 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 11:06:50.921463 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 11:06:50.922580 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 11:06:50.922580 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 11:06:50.922580 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 11:06:50.922580 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 11:06:50.922580 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:06:50.922580 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:06:50.922580 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:06:50.922580 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:06:50.922580 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:06:50.922580 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:06:50.922580 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:06:50.922580 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jun 29 11:06:50.922580 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:06:50.922580 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:06:50.922580 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:06:50.922580 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:06:50.923445 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:06:50.923445 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:06:50.923445 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:06:50.923445 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:06:50.923445 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 11:06:50.923445 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 11:06:50.923445 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 11:06:50.923445 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 11:06:50.923445 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 11:06:50.923445 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 11:06:50.923445 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 29 11:06:50.923445 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:06:50.923445 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 288/582] 10.222.0.44 () {56 vars in 1102 bytes} [Wed Jun 29 11:06:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:06:50.924173 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c4cf33-b7dc-4f43-b49b-86e3101ad224 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:50.983063 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7aba919-fff0-4284-a9da-045741585b84 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:51.015376 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130c69f9-364d-4136-810d-56a04fc574c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:51.016062 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130c69f9-364d-4136-810d-56a04fc574c9 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:51.016346 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130c69f9-364d-4136-810d-56a04fc574c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:51.016623 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130c69f9-364d-4136-810d-56a04fc574c9 None admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:51.017332 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130c69f9-364d-4136-810d-56a04fc574c9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:06:51.017636 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130c69f9-364d-4136-810d-56a04fc574c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:51.018993 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130c69f9-364d-4136-810d-56a04fc574c9 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:51.020977 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:51.022154 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:51.026178 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c4cf33-b7dc-4f43-b49b-86e3101ad224 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:51.031376 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 295/583] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Jun 29 11:06:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:51.034173 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130c69f9-364d-4136-810d-56a04fc574c9 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:51.059879 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354863fe-f8c2-41d8-86cc-ddbb08ea2f30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:51.060534 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354863fe-f8c2-41d8-86cc-ddbb08ea2f30 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:51.060821 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354863fe-f8c2-41d8-86cc-ddbb08ea2f30 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:51.061099 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354863fe-f8c2-41d8-86cc-ddbb08ea2f30 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:51.068782 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354863fe-f8c2-41d8-86cc-ddbb08ea2f30 None admin] RBAC: Adding query filter params (role.id=a8212be3426741cbb1b38b1646c66405, scope.project.id=ebe5dffa4bb8440286687daefb6c98f4, user.id=e3b38493ba904759badc800e396befda) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:06:51.068782 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354863fe-f8c2-41d8-86cc-ddbb08ea2f30 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:51.068782 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354863fe-f8c2-41d8-86cc-ddbb08ea2f30 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:51.069438 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 289/584] 10.222.0.44 () {56 vars in 1374 bytes} [Wed Jun 29 11:06:50 2022] GET /identity/v3/role_assignments?user.id=e3b38493ba904759badc800e396befda&scope.project.id=ebe5dffa4bb8440286687daefb6c98f4&role.id=a8212be3426741cbb1b38b1646c66405 => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:51.070180 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354863fe-f8c2-41d8-86cc-ddbb08ea2f30 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:51.141388 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130c69f9-364d-4136-810d-56a04fc574c9 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:51.178935 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54fc0cc-d131-4f98-aa5f-3325648444ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:51.179889 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54fc0cc-d131-4f98-aa5f-3325648444ff None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:51.180242 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54fc0cc-d131-4f98-aa5f-3325648444ff None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:51.180601 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54fc0cc-d131-4f98-aa5f-3325648444ff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:51.192835 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54fc0cc-d131-4f98-aa5f-3325648444ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:51.194305 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54fc0cc-d131-4f98-aa5f-3325648444ff None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:51.197819 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354863fe-f8c2-41d8-86cc-ddbb08ea2f30 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:51.213366 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-a54fc0cc-d131-4f98-aa5f-3325648444ff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 11:06:51.213366 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:06:51.213366 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:06:51.213366 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:06:51.213366 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:06:51.213366 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:06:51.213366 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:06:51.213366 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:06:51.213366 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:06:51.213366 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:06:51.213366 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:06:51.213366 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:06:51.213366 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 11:06:51.213366 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 11:06:51.213366 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 11:06:51.214358 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 11:06:51.214358 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:06:51.214358 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:06:51.214358 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:06:51.214358 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:06:51.214358 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:06:51.214358 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:06:51.214358 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:06:51.214358 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jun 29 11:06:51.214358 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:06:51.214358 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:06:51.214358 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:06:51.214358 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:06:51.214358 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:06:51.214358 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:06:51.214358 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:06:51.214358 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:06:51.216188 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 11:06:51.216188 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 11:06:51.216188 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 11:06:51.216188 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jun 29 11:06:51.216188 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 296/585] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jun 29 11:06:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:06:51.216188 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54fc0cc-d131-4f98-aa5f-3325648444ff None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:51.241290 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d44b2a-9185-4d52-96cc-1488192a0d7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:51.242117 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d44b2a-9185-4d52-96cc-1488192a0d7a None admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:51.242401 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d44b2a-9185-4d52-96cc-1488192a0d7a None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:51.242678 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d44b2a-9185-4d52-96cc-1488192a0d7a None admin] PATH_INFO: `/v3/projects/ebe5dffa4bb8440286687daefb6c98f4/users/796aeeff16df4739bddec1f6dee61bfe/roles/a8212be3426741cbb1b38b1646c66405` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:51.257975 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d44b2a-9185-4d52-96cc-1488192a0d7a None admin] RBAC: Authorizing `identity:create_grant(project_id=ebe5dffa4bb8440286687daefb6c98f4, user_id=796aeeff16df4739bddec1f6dee61bfe, role_id=a8212be3426741cbb1b38b1646c66405)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:51.259403 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d44b2a-9185-4d52-96cc-1488192a0d7a None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:51.266729 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:51.322130 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54fc0cc-d131-4f98-aa5f-3325648444ff None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:51.332084 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 290/586] 10.222.0.44 () {58 vars in 1439 bytes} [Wed Jun 29 11:06:51 2022] PUT /identity/v3/projects/ebe5dffa4bb8440286687daefb6c98f4/users/796aeeff16df4739bddec1f6dee61bfe/roles/a8212be3426741cbb1b38b1646c66405 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:06:51.332084 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d44b2a-9185-4d52-96cc-1488192a0d7a None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:51.340617 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:51.379393 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fabc4b2-2e12-49e8-a90d-2cbbed91a6b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:51.381011 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fabc4b2-2e12-49e8-a90d-2cbbed91a6b1 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:51.381369 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fabc4b2-2e12-49e8-a90d-2cbbed91a6b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:51.381887 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fabc4b2-2e12-49e8-a90d-2cbbed91a6b1 None admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:51.383084 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fabc4b2-2e12-49e8-a90d-2cbbed91a6b1 None admin] RBAC: Adding query filter params (name=service) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:06:51.383551 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fabc4b2-2e12-49e8-a90d-2cbbed91a6b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:51.385118 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fabc4b2-2e12-49e8-a90d-2cbbed91a6b1 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:51.397653 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 297/587] 10.222.0.44 () {56 vars in 1103 bytes} [Wed Jun 29 11:06:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:51.398683 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fabc4b2-2e12-49e8-a90d-2cbbed91a6b1 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:51.448979 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d44b2a-9185-4d52-96cc-1488192a0d7a None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:51.488143 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ec2aad-a78d-4a6d-949c-14d9e0fddd35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:51.489490 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ec2aad-a78d-4a6d-949c-14d9e0fddd35 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:51.489782 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ec2aad-a78d-4a6d-949c-14d9e0fddd35 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:51.490118 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ec2aad-a78d-4a6d-949c-14d9e0fddd35 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:51.499963 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ec2aad-a78d-4a6d-949c-14d9e0fddd35 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:51.501360 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ec2aad-a78d-4a6d-949c-14d9e0fddd35 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:51.518315 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-43ec2aad-a78d-4a6d-949c-14d9e0fddd35 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 29 11:06:51.518315 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:06:51.518315 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:06:51.518315 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:06:51.518315 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:06:51.518315 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:06:51.518315 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:06:51.518315 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:06:51.518315 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:06:51.518315 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:06:51.518315 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:06:51.518315 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:06:51.518315 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 11:06:51.518315 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 11:06:51.518315 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 11:06:51.518315 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 11:06:51.519294 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:06:51.519294 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:06:51.519294 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 11:06:51.519294 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 11:06:51.519294 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 11:06:51.519294 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 11:06:51.519294 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:06:51.519294 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:06:51.519294 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:06:51.519294 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:06:51.519294 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:06:51.519294 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jun 29 11:06:51.519294 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:06:51.519294 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:06:51.519294 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:06:51.519294 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:06:51.519294 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:06:51.519294 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:06:51.520335 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:06:51.520335 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:06:51.520335 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 11:06:51.520335 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 11:06:51.520335 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 11:06:51.520335 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 11:06:51.520335 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 11:06:51.520335 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 11:06:51.520335 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 29 11:06:51.520335 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:06:51.520335 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 291/588] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Jun 29 11:06:51 2022] GET /identity/v3/users/cinder => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:06:51.521001 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ec2aad-a78d-4a6d-949c-14d9e0fddd35 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:51.537084 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fabc4b2-2e12-49e8-a90d-2cbbed91a6b1 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:51.548044 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:51.582849 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638f6201-77a5-40ba-8bcb-0af301895b81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:51.584315 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638f6201-77a5-40ba-8bcb-0af301895b81 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:51.585009 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638f6201-77a5-40ba-8bcb-0af301895b81 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:51.585458 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638f6201-77a5-40ba-8bcb-0af301895b81 None admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:51.586625 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638f6201-77a5-40ba-8bcb-0af301895b81 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:06:51.587474 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638f6201-77a5-40ba-8bcb-0af301895b81 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:51.589082 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638f6201-77a5-40ba-8bcb-0af301895b81 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:51.606857 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 298/589] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Jun 29 11:06:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:51.608315 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638f6201-77a5-40ba-8bcb-0af301895b81 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:51.609621 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:51.641878 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ec2aad-a78d-4a6d-949c-14d9e0fddd35 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:51.688440 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3020b6b-428e-4718-98c2-f4ff1e365917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:51.689535 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3020b6b-428e-4718-98c2-f4ff1e365917 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:51.690218 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3020b6b-428e-4718-98c2-f4ff1e365917 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:51.690696 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3020b6b-428e-4718-98c2-f4ff1e365917 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:51.705708 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3020b6b-428e-4718-98c2-f4ff1e365917 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:51.707340 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3020b6b-428e-4718-98c2-f4ff1e365917 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:51.726767 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-a3020b6b-428e-4718-98c2-f4ff1e365917 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 11:06:51.726767 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:06:51.726767 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:06:51.726767 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:06:51.726767 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:06:51.726767 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:06:51.726767 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:06:51.726767 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:06:51.726767 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:06:51.726767 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:06:51.726767 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:06:51.726767 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:06:51.726767 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 11:06:51.726767 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 11:06:51.726767 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 11:06:51.727781 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 11:06:51.727781 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:06:51.727781 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:06:51.727781 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:06:51.727781 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:06:51.727781 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:06:51.727781 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:06:51.727781 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:06:51.727781 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jun 29 11:06:51.727781 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:06:51.727781 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:06:51.727781 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:06:51.727781 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:06:51.727781 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:06:51.727781 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:06:51.727781 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:06:51.727781 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:06:51.728727 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 11:06:51.728727 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 11:06:51.728727 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 11:06:51.728727 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:06:51.728727 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 292/590] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jun 29 11:06:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:06:51.728727 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3020b6b-428e-4718-98c2-f4ff1e365917 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:51.736532 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638f6201-77a5-40ba-8bcb-0af301895b81 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:51.777944 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92447a86-24d4-4768-ad40-83e93be95c1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:51.778871 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92447a86-24d4-4768-ad40-83e93be95c1d None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:51.779378 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92447a86-24d4-4768-ad40-83e93be95c1d None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:51.779803 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92447a86-24d4-4768-ad40-83e93be95c1d None admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:51.780802 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92447a86-24d4-4768-ad40-83e93be95c1d None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:06:51.781250 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92447a86-24d4-4768-ad40-83e93be95c1d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:51.783028 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92447a86-24d4-4768-ad40-83e93be95c1d None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:51.802486 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 299/591] 10.222.0.44 () {56 vars in 1137 bytes} [Wed Jun 29 11:06:51 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:51.803723 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92447a86-24d4-4768-ad40-83e93be95c1d None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:51.833943 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:51.840154 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3020b6b-428e-4718-98c2-f4ff1e365917 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:51.864687 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:51.879064 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b161fdba-fbf9-4e3d-8552-b28b53c5e223 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:51.880428 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b161fdba-fbf9-4e3d-8552-b28b53c5e223 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:51.880714 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b161fdba-fbf9-4e3d-8552-b28b53c5e223 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:51.881098 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b161fdba-fbf9-4e3d-8552-b28b53c5e223 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:51.888471 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b161fdba-fbf9-4e3d-8552-b28b53c5e223 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:51.890359 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b161fdba-fbf9-4e3d-8552-b28b53c5e223 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:51.910225 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-b161fdba-fbf9-4e3d-8552-b28b53c5e223 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 11:06:51.910225 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:06:51.910225 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:06:51.910225 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:06:51.910225 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:06:51.910225 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:06:51.910225 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:06:51.910225 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:06:51.910225 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:06:51.910225 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:06:51.910225 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:06:51.910225 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:06:51.910225 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 11:06:51.910225 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 11:06:51.910225 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 11:06:51.911291 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 11:06:51.911291 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:06:51.911291 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:06:51.911291 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:06:51.911291 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:06:51.911291 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:06:51.911291 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:06:51.911291 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:06:51.911291 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jun 29 11:06:51.911291 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:06:51.911291 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:06:51.911291 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:06:51.911291 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:06:51.911291 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:06:51.911291 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:06:51.911291 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:06:51.911291 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:06:51.912576 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 11:06:51.912576 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 11:06:51.912576 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 11:06:51.912576 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 11:06:51.912576 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 11:06:51.912576 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 11:06:51.912576 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 11:06:51.912576 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:06:51.913107 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 293/592] 10.222.0.44 () {56 vars in 1111 bytes} [Wed Jun 29 11:06:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:06:51.916895 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b161fdba-fbf9-4e3d-8552-b28b53c5e223 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:51.927271 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92447a86-24d4-4768-ad40-83e93be95c1d None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:51.967544 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a956a86-92ff-49ea-b593-71d2d9e74b3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:51.968342 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a956a86-92ff-49ea-b593-71d2d9e74b3c None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:51.968694 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a956a86-92ff-49ea-b593-71d2d9e74b3c None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:51.969042 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a956a86-92ff-49ea-b593-71d2d9e74b3c None admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:51.969859 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a956a86-92ff-49ea-b593-71d2d9e74b3c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:06:51.970236 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a956a86-92ff-49ea-b593-71d2d9e74b3c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:51.971678 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a956a86-92ff-49ea-b593-71d2d9e74b3c None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:51.987737 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 300/593] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Jun 29 11:06:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:51.988685 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a956a86-92ff-49ea-b593-71d2d9e74b3c None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:52.040837 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b161fdba-fbf9-4e3d-8552-b28b53c5e223 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:52.082853 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de0835e-fcf7-4a35-8300-67f6934006a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:52.083755 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de0835e-fcf7-4a35-8300-67f6934006a6 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:52.084137 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de0835e-fcf7-4a35-8300-67f6934006a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:52.084580 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de0835e-fcf7-4a35-8300-67f6934006a6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:52.089455 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:52.097617 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de0835e-fcf7-4a35-8300-67f6934006a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:52.099491 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de0835e-fcf7-4a35-8300-67f6934006a6 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:52.106823 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a956a86-92ff-49ea-b593-71d2d9e74b3c None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:52.118236 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-8de0835e-fcf7-4a35-8300-67f6934006a6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 11:06:52.118236 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:06:52.118236 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:06:52.118236 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:06:52.118236 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:06:52.118236 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:06:52.118236 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:06:52.118236 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:06:52.118236 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:06:52.118236 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:06:52.118236 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:06:52.118236 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:06:52.118236 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 11:06:52.118236 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 11:06:52.118236 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 11:06:52.119294 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 11:06:52.119294 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:06:52.119294 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:06:52.119294 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:06:52.119294 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:06:52.119294 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:06:52.119294 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:06:52.119294 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:06:52.119294 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jun 29 11:06:52.119294 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:06:52.119294 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:06:52.119294 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:06:52.119294 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:06:52.119294 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:06:52.119294 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:06:52.119294 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:06:52.119294 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:06:52.120320 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 11:06:52.120320 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 11:06:52.120320 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 11:06:52.120320 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:06:52.122108 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 294/594] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jun 29 11:06:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:06:52.123023 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de0835e-fcf7-4a35-8300-67f6934006a6 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:52.153516 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c73a0a5-2f6f-4cc1-9c1a-6ba30d4fe9a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:52.154581 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c73a0a5-2f6f-4cc1-9c1a-6ba30d4fe9a5 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:52.154873 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c73a0a5-2f6f-4cc1-9c1a-6ba30d4fe9a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:52.155162 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c73a0a5-2f6f-4cc1-9c1a-6ba30d4fe9a5 None admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:52.156236 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c73a0a5-2f6f-4cc1-9c1a-6ba30d4fe9a5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:06:52.156553 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c73a0a5-2f6f-4cc1-9c1a-6ba30d4fe9a5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:52.158553 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c73a0a5-2f6f-4cc1-9c1a-6ba30d4fe9a5 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:52.161963 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:52.174905 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 301/595] 10.222.0.44 () {56 vars in 1148 bytes} [Wed Jun 29 11:06:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:52.174905 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c73a0a5-2f6f-4cc1-9c1a-6ba30d4fe9a5 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:52.252663 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de0835e-fcf7-4a35-8300-67f6934006a6 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:52.295194 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde445e6-73e6-4b57-b069-d0c81a5c163e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:52.295942 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde445e6-73e6-4b57-b069-d0c81a5c163e None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:52.296233 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde445e6-73e6-4b57-b069-d0c81a5c163e None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:52.296516 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde445e6-73e6-4b57-b069-d0c81a5c163e None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:52.307247 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c73a0a5-2f6f-4cc1-9c1a-6ba30d4fe9a5 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:52.307432 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde445e6-73e6-4b57-b069-d0c81a5c163e None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:52.308888 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde445e6-73e6-4b57-b069-d0c81a5c163e None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:52.329087 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-cde445e6-73e6-4b57-b069-d0c81a5c163e None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 29 11:06:52.329087 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:06:52.329087 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:06:52.329087 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:06:52.329087 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:06:52.329087 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:06:52.329087 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:06:52.329087 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:06:52.329087 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:06:52.329087 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:06:52.329087 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:06:52.329087 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:06:52.329087 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 11:06:52.329087 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 11:06:52.329087 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 11:06:52.329087 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 11:06:52.330060 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:06:52.330060 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:06:52.330060 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 11:06:52.330060 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 11:06:52.330060 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 11:06:52.330060 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 11:06:52.330060 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:06:52.330060 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:06:52.330060 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:06:52.330060 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:06:52.330060 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:06:52.330060 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jun 29 11:06:52.330060 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:06:52.330060 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:06:52.330060 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:06:52.330060 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:06:52.330060 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:06:52.330060 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:06:52.330994 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:06:52.330994 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:06:52.330994 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 11:06:52.330994 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 11:06:52.330994 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 11:06:52.330994 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 11:06:52.330994 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 11:06:52.330994 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 11:06:52.330994 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 29 11:06:52.330994 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:06:52.330994 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 295/596] 10.222.0.44 () {56 vars in 1102 bytes} [Wed Jun 29 11:06:52 2022] GET /identity/v3/users/neutron => generated 85 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:06:52.330994 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde445e6-73e6-4b57-b069-d0c81a5c163e None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:52.353073 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9454d1ea-5ac1-46a4-84d4-60d907044a99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:52.353924 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9454d1ea-5ac1-46a4-84d4-60d907044a99 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:52.354301 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9454d1ea-5ac1-46a4-84d4-60d907044a99 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:52.354698 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9454d1ea-5ac1-46a4-84d4-60d907044a99 None admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:52.355570 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9454d1ea-5ac1-46a4-84d4-60d907044a99 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:06:52.356020 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9454d1ea-5ac1-46a4-84d4-60d907044a99 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:52.357485 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9454d1ea-5ac1-46a4-84d4-60d907044a99 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:52.371453 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 302/597] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Jun 29 11:06:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:52.372491 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9454d1ea-5ac1-46a4-84d4-60d907044a99 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:52.387505 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:52.442001 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:52.449066 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde445e6-73e6-4b57-b069-d0c81a5c163e None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:52.492424 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273fd906-16e1-4248-845b-31f545dd01be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:52.493929 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273fd906-16e1-4248-845b-31f545dd01be None admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:52.494876 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273fd906-16e1-4248-845b-31f545dd01be None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:52.495359 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273fd906-16e1-4248-845b-31f545dd01be None admin] PATH_INFO: `/v3/projects/ebe5dffa4bb8440286687daefb6c98f4/users/ede3a17c7063432e8ee66462766584f8/roles/a8212be3426741cbb1b38b1646c66405` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:52.510553 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9454d1ea-5ac1-46a4-84d4-60d907044a99 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:52.511676 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273fd906-16e1-4248-845b-31f545dd01be None admin] RBAC: Authorizing `identity:create_grant(project_id=ebe5dffa4bb8440286687daefb6c98f4, user_id=ede3a17c7063432e8ee66462766584f8, role_id=a8212be3426741cbb1b38b1646c66405)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:52.513616 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273fd906-16e1-4248-845b-31f545dd01be None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:52.547949 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2f4b6e-c36f-4b83-ae5c-48828c18e682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:52.548539 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2f4b6e-c36f-4b83-ae5c-48828c18e682 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:52.548820 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2f4b6e-c36f-4b83-ae5c-48828c18e682 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:52.549099 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2f4b6e-c36f-4b83-ae5c-48828c18e682 None admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:52.549928 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2f4b6e-c36f-4b83-ae5c-48828c18e682 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:06:52.550236 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2f4b6e-c36f-4b83-ae5c-48828c18e682 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:52.552287 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2f4b6e-c36f-4b83-ae5c-48828c18e682 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:52.576083 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 303/598] 10.222.0.44 () {56 vars in 1139 bytes} [Wed Jun 29 11:06:52 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:52.576083 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2f4b6e-c36f-4b83-ae5c-48828c18e682 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:52.593715 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 296/599] 10.222.0.44 () {58 vars in 1439 bytes} [Wed Jun 29 11:06:52 2022] PUT /identity/v3/projects/ebe5dffa4bb8440286687daefb6c98f4/users/ede3a17c7063432e8ee66462766584f8/roles/a8212be3426741cbb1b38b1646c66405 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:06:52.597499 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273fd906-16e1-4248-845b-31f545dd01be None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:52.719428 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:52.790178 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2f4b6e-c36f-4b83-ae5c-48828c18e682 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:52.820634 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273fd906-16e1-4248-845b-31f545dd01be None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:52.828530 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6354ed00-6bb5-498b-9912-d474c14c9b79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:52.829336 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6354ed00-6bb5-498b-9912-d474c14c9b79 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:52.829615 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6354ed00-6bb5-498b-9912-d474c14c9b79 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:52.829886 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6354ed00-6bb5-498b-9912-d474c14c9b79 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:52.837330 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6354ed00-6bb5-498b-9912-d474c14c9b79 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:52.845037 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6354ed00-6bb5-498b-9912-d474c14c9b79 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:52.861900 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-6354ed00-6bb5-498b-9912-d474c14c9b79 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 11:06:52.861900 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:06:52.861900 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:06:52.861900 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:06:52.861900 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:06:52.861900 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:06:52.861900 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:06:52.861900 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:06:52.861900 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:06:52.861900 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:06:52.861900 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:06:52.861900 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:06:52.861900 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 11:06:52.861900 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 11:06:52.861900 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 11:06:52.862826 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 11:06:52.862826 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:06:52.862826 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:06:52.862826 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:06:52.862826 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:06:52.862826 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:06:52.862826 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:06:52.862826 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:06:52.862826 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jun 29 11:06:52.862826 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:06:52.862826 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:06:52.862826 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:06:52.862826 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:06:52.862826 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:06:52.862826 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:06:52.862826 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:06:52.862826 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:06:52.863728 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 11:06:52.863728 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 11:06:52.863728 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 11:06:52.863728 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 11:06:52.863728 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 11:06:52.863728 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 11:06:52.863728 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 11:06:52.863728 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jun 29 11:06:52.863728 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 304/600] 10.222.0.44 () {56 vars in 1111 bytes} [Wed Jun 29 11:06:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:06:52.866598 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc8b396-b902-4576-9f60-0ab5f8e927e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:52.867554 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc8b396-b902-4576-9f60-0ab5f8e927e3 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:52.868016 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc8b396-b902-4576-9f60-0ab5f8e927e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:52.868450 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc8b396-b902-4576-9f60-0ab5f8e927e3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:52.869574 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6354ed00-6bb5-498b-9912-d474c14c9b79 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:52.881964 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc8b396-b902-4576-9f60-0ab5f8e927e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:52.884188 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc8b396-b902-4576-9f60-0ab5f8e927e3 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:52.892844 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:52.905119 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-5fc8b396-b902-4576-9f60-0ab5f8e927e3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 11:06:52.905119 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:06:52.905119 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:06:52.905119 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:06:52.905119 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:06:52.905119 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:06:52.905119 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:06:52.905119 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:06:52.905119 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:06:52.905119 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:06:52.905119 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:06:52.905119 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:06:52.905119 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 11:06:52.905119 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 11:06:52.905119 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 11:06:52.906012 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 11:06:52.906012 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:06:52.906012 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:06:52.906012 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:06:52.906012 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:06:52.906012 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:06:52.906012 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:06:52.906012 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:06:52.906012 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jun 29 11:06:52.906012 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:06:52.906012 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:06:52.906012 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:06:52.906012 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:06:52.906012 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:06:52.906012 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:06:52.906012 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:06:52.906012 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:06:52.906900 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 11:06:52.906900 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 11:06:52.906900 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 11:06:52.906900 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:06:52.909239 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 297/601] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jun 29 11:06:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:06:52.915195 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc8b396-b902-4576-9f60-0ab5f8e927e3 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:53.048450 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6354ed00-6bb5-498b-9912-d474c14c9b79 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:53.084326 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1585c4-8827-410f-b1e7-1ee5131186f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:53.085348 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1585c4-8827-410f-b1e7-1ee5131186f7 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:53.085856 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1585c4-8827-410f-b1e7-1ee5131186f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:53.086329 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1585c4-8827-410f-b1e7-1ee5131186f7 None admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:53.087309 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1585c4-8827-410f-b1e7-1ee5131186f7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:06:53.087821 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1585c4-8827-410f-b1e7-1ee5131186f7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:53.089509 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1585c4-8827-410f-b1e7-1ee5131186f7 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:53.103031 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:53.103625 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 305/602] 10.222.0.44 () {56 vars in 1148 bytes} [Wed Jun 29 11:06:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:53.109647 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1585c4-8827-410f-b1e7-1ee5131186f7 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:53.123155 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc8b396-b902-4576-9f60-0ab5f8e927e3 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:53.188670 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cd4d70-245e-4954-b2d2-8756e531a6b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:53.189340 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cd4d70-245e-4954-b2d2-8756e531a6b6 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:53.189620 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cd4d70-245e-4954-b2d2-8756e531a6b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:53.189894 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cd4d70-245e-4954-b2d2-8756e531a6b6 None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:53.190613 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cd4d70-245e-4954-b2d2-8756e531a6b6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:06:53.190917 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cd4d70-245e-4954-b2d2-8756e531a6b6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:53.192875 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cd4d70-245e-4954-b2d2-8756e531a6b6 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:53.214922 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 298/603] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Jun 29 11:06:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 11:06:53.222207 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cd4d70-245e-4954-b2d2-8756e531a6b6 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:53.272157 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1585c4-8827-410f-b1e7-1ee5131186f7 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:53.296039 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:53.310817 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b13fae7-5504-4070-a1a3-017befb5e5c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:53.312088 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b13fae7-5504-4070-a1a3-017befb5e5c8 None admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:53.312630 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b13fae7-5504-4070-a1a3-017befb5e5c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:53.313106 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b13fae7-5504-4070-a1a3-017befb5e5c8 None admin] PATH_INFO: `/v3/projects/ebe5dffa4bb8440286687daefb6c98f4/users/ff7931b75a924394b1af733febce0834/roles/a8212be3426741cbb1b38b1646c66405` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:53.330111 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b13fae7-5504-4070-a1a3-017befb5e5c8 None admin] RBAC: Authorizing `identity:create_grant(project_id=ebe5dffa4bb8440286687daefb6c98f4, user_id=ff7931b75a924394b1af733febce0834, role_id=a8212be3426741cbb1b38b1646c66405)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:53.333424 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b13fae7-5504-4070-a1a3-017befb5e5c8 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:53.341073 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cd4d70-245e-4954-b2d2-8756e531a6b6 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:53.379113 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897ead72-dde9-4a26-84ea-3dbd0147f18b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:53.380780 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897ead72-dde9-4a26-84ea-3dbd0147f18b None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:53.381152 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897ead72-dde9-4a26-84ea-3dbd0147f18b None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:53.381475 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897ead72-dde9-4a26-84ea-3dbd0147f18b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:53.389088 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897ead72-dde9-4a26-84ea-3dbd0147f18b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:53.390585 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897ead72-dde9-4a26-84ea-3dbd0147f18b None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:53.405722 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-897ead72-dde9-4a26-84ea-3dbd0147f18b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 11:06:53.405722 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:06:53.405722 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:06:53.405722 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:06:53.405722 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:06:53.405722 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:06:53.405722 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:06:53.405722 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:06:53.405722 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:06:53.405722 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:06:53.405722 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:06:53.405722 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:06:53.405722 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 11:06:53.405722 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 11:06:53.405722 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 11:06:53.407097 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 11:06:53.407097 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:06:53.407097 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:06:53.407097 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:06:53.407097 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:06:53.407097 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:06:53.407097 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:06:53.407097 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:06:53.407097 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jun 29 11:06:53.407097 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:06:53.407097 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:06:53.407097 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:06:53.407097 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:06:53.407097 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:06:53.407097 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:06:53.407097 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:06:53.407097 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:06:53.408062 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 11:06:53.408062 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 11:06:53.408062 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 11:06:53.408062 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 11:06:53.408062 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 11:06:53.408062 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 11:06:53.408062 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 11:06:53.408062 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:06:53.408062 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 299/604] 10.222.0.44 () {56 vars in 1111 bytes} [Wed Jun 29 11:06:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:06:53.408582 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c36a180-10e7-41f8-a461-c818adcd49b1 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:53.414729 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c36a180-10e7-41f8-a461-c818adcd49b1 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:53.414729 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c36a180-10e7-41f8-a461-c818adcd49b1 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:53.414729 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 300/605] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:06:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:06:53.414729 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c36a180-10e7-41f8-a461-c818adcd49b1 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:53.435427 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 306/606] 10.222.0.44 () {58 vars in 1439 bytes} [Wed Jun 29 11:06:53 2022] PUT /identity/v3/projects/ebe5dffa4bb8440286687daefb6c98f4/users/ff7931b75a924394b1af733febce0834/roles/a8212be3426741cbb1b38b1646c66405 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:06:53.437482 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23d9791-9b6d-40fd-b122-8bfd80ec3476 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:53.438010 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23d9791-9b6d-40fd-b122-8bfd80ec3476 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:53.438503 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23d9791-9b6d-40fd-b122-8bfd80ec3476 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:53.461892 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-d23d9791-9b6d-40fd-b122-8bfd80ec3476 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:06:53.463659 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-d23d9791-9b6d-40fd-b122-8bfd80ec3476 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:06:53.501499 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:53.553231 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 307/607] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:06:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 11:06:53.599268 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23d9791-9b6d-40fd-b122-8bfd80ec3476 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:53.621394 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c36a180-10e7-41f8-a461-c818adcd49b1 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:53.643319 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:53.666737 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8d4d7c-aa4e-4546-bf07-efaa6a1c87c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:53.667586 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8d4d7c-aa4e-4546-bf07-efaa6a1c87c7 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:53.667915 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8d4d7c-aa4e-4546-bf07-efaa6a1c87c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:53.668205 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8d4d7c-aa4e-4546-bf07-efaa6a1c87c7 None admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:53.669309 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8d4d7c-aa4e-4546-bf07-efaa6a1c87c7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:06:53.669632 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8d4d7c-aa4e-4546-bf07-efaa6a1c87c7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:53.671103 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8d4d7c-aa4e-4546-bf07-efaa6a1c87c7 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:53.688642 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 301/608] 10.222.0.44 () {56 vars in 1148 bytes} [Wed Jun 29 11:06:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:53.690765 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d295efb-8fb3-496c-843f-a26c4fa98c83 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:53.691060 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d295efb-8fb3-496c-843f-a26c4fa98c83 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:53.691345 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d295efb-8fb3-496c-843f-a26c4fa98c83 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:53.693173 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 302/609] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:06:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:06:53.696882 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d295efb-8fb3-496c-843f-a26c4fa98c83 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:53.732453 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23d9791-9b6d-40fd-b122-8bfd80ec3476 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:53.779573 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7627ee6-a556-4534-bbec-7b5acb6cc223 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:53.780934 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7627ee6-a556-4534-bbec-7b5acb6cc223 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:53.781221 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7627ee6-a556-4534-bbec-7b5acb6cc223 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:53.781500 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7627ee6-a556-4534-bbec-7b5acb6cc223 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:53.795844 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7627ee6-a556-4534-bbec-7b5acb6cc223 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 11:06:53.807390 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-d7627ee6-a556-4534-bbec-7b5acb6cc223 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 29 11:06:53.807390 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:06:53.807390 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:06:53.807390 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:06:53.807390 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:06:53.807390 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:06:53.807390 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:06:53.807390 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:06:53.807390 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:06:53.807390 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:06:53.807390 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:06:53.807390 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:06:53.807390 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 11:06:53.807390 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 11:06:53.807390 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 11:06:53.807390 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 11:06:53.808449 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 11:06:53.808449 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 11:06:53.808449 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 11:06:53.808449 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 11:06:53.808449 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:06:53.808449 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:06:53.808449 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:06:53.808449 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:06:53.808449 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:06:53.808449 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:06:53.808449 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:06:53.808449 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jun 29 11:06:53.808449 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:06:53.808449 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:06:53.808449 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:06:53.808449 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:06:53.809285 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:06:53.809285 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:06:53.809285 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:06:53.809285 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:06:53.809285 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 11:06:53.809285 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 11:06:53.809285 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 11:06:53.809285 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 11:06:53.809285 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 11:06:53.809285 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 11:06:53.809285 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 29 11:06:53.809285 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jun 29 11:06:53.809914 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d295efb-8fb3-496c-843f-a26c4fa98c83 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:53.809996 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 308/610] 10.222.0.44 () {56 vars in 1102 bytes} [Wed Jun 29 11:06:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 11:06:53.810695 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42336431-60d4-4deb-92f6-efab1ec0ee5e None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:53.810695 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42336431-60d4-4deb-92f6-efab1ec0ee5e None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:53.810978 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42336431-60d4-4deb-92f6-efab1ec0ee5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:53.837991 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-42336431-60d4-4deb-92f6-efab1ec0ee5e None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:06:53.840252 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-42336431-60d4-4deb-92f6-efab1ec0ee5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:06:53.861280 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:53.862866 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2a0ee1-ef6b-4b0f-b0f4-e01345831e18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:53.864916 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2a0ee1-ef6b-4b0f-b0f4-e01345831e18 None admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:53.865383 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2a0ee1-ef6b-4b0f-b0f4-e01345831e18 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:53.866305 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2a0ee1-ef6b-4b0f-b0f4-e01345831e18 None admin] PATH_INFO: `/v3/projects/ebe5dffa4bb8440286687daefb6c98f4/users/b5f4bb9f989e4024a5da28dced646b92/roles/a8212be3426741cbb1b38b1646c66405` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:53.886044 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2a0ee1-ef6b-4b0f-b0f4-e01345831e18 None admin] RBAC: Authorizing `identity:create_grant(project_id=ebe5dffa4bb8440286687daefb6c98f4, user_id=b5f4bb9f989e4024a5da28dced646b92, role_id=a8212be3426741cbb1b38b1646c66405)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:53.888281 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2a0ee1-ef6b-4b0f-b0f4-e01345831e18 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:53.936157 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 309/611] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:06:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 11:06:53.936157 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42336431-60d4-4deb-92f6-efab1ec0ee5e None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:53.965841 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 303/612] 10.222.0.44 () {58 vars in 1439 bytes} [Wed Jun 29 11:06:53 2022] PUT /identity/v3/projects/ebe5dffa4bb8440286687daefb6c98f4/users/b5f4bb9f989e4024a5da28dced646b92/roles/a8212be3426741cbb1b38b1646c66405 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:06:53.966710 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2a0ee1-ef6b-4b0f-b0f4-e01345831e18 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:53.996994 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:54.055575 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42336431-60d4-4deb-92f6-efab1ec0ee5e None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:54.101058 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123d8c2f-4f8d-434d-b9a5-658ddf9a7366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:54.102207 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123d8c2f-4f8d-434d-b9a5-658ddf9a7366 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:54.102954 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123d8c2f-4f8d-434d-b9a5-658ddf9a7366 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:54.103381 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123d8c2f-4f8d-434d-b9a5-658ddf9a7366 None admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:54.104320 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123d8c2f-4f8d-434d-b9a5-658ddf9a7366 None admin] RBAC: Adding query filter params (name=service) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:06:54.104780 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123d8c2f-4f8d-434d-b9a5-658ddf9a7366 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:54.106322 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123d8c2f-4f8d-434d-b9a5-658ddf9a7366 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:54.117028 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 310/613] 10.222.0.44 () {56 vars in 1103 bytes} [Wed Jun 29 11:06:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:54.123195 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123d8c2f-4f8d-434d-b9a5-658ddf9a7366 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:54.148440 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:54.174084 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2a0ee1-ef6b-4b0f-b0f4-e01345831e18 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:54.252757 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123d8c2f-4f8d-434d-b9a5-658ddf9a7366 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:54.257935 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0f1576-bd40-472c-84e2-f736036ce950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:54.258748 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0f1576-bd40-472c-84e2-f736036ce950 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:54.259030 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0f1576-bd40-472c-84e2-f736036ce950 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:54.259306 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0f1576-bd40-472c-84e2-f736036ce950 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:54.273711 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0f1576-bd40-472c-84e2-f736036ce950 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 11:06:54.283892 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-3e0f1576-bd40-472c-84e2-f736036ce950 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 29 11:06:54.283892 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:06:54.283892 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:06:54.283892 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:06:54.283892 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:06:54.283892 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:06:54.283892 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:06:54.283892 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:06:54.283892 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:06:54.283892 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:06:54.283892 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:06:54.283892 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:06:54.283892 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 11:06:54.283892 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 11:06:54.283892 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 11:06:54.283892 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 11:06:54.284801 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 11:06:54.284801 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 11:06:54.284801 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 11:06:54.284801 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 11:06:54.284801 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:06:54.284801 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:06:54.284801 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:06:54.284801 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:06:54.284801 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:06:54.284801 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:06:54.284801 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:06:54.284801 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jun 29 11:06:54.284801 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:06:54.284801 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:06:54.284801 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:06:54.284801 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:06:54.285641 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:06:54.285641 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:06:54.285641 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:06:54.285641 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:06:54.285641 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 11:06:54.285641 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 11:06:54.285641 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 11:06:54.285641 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 11:06:54.285641 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 11:06:54.285641 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 11:06:54.285641 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 29 11:06:54.285641 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:06:54.286323 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 304/614] 10.222.0.44 () {56 vars in 1102 bytes} [Wed Jun 29 11:06:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:06:54.291653 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0f1576-bd40-472c-84e2-f736036ce950 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:54.292181 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c26a042-d650-4ee0-8c3d-f3cc08e426a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:54.292938 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c26a042-d650-4ee0-8c3d-f3cc08e426a8 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:54.293223 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c26a042-d650-4ee0-8c3d-f3cc08e426a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:54.293554 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c26a042-d650-4ee0-8c3d-f3cc08e426a8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:54.306461 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c26a042-d650-4ee0-8c3d-f3cc08e426a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:54.307954 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c26a042-d650-4ee0-8c3d-f3cc08e426a8 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:54.327368 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-1c26a042-d650-4ee0-8c3d-f3cc08e426a8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 11:06:54.327368 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:06:54.327368 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:06:54.327368 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:06:54.327368 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:06:54.327368 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:06:54.327368 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:06:54.327368 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:06:54.327368 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:06:54.327368 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:06:54.327368 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:06:54.327368 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:06:54.327368 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 11:06:54.327368 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 11:06:54.327368 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 11:06:54.329451 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 11:06:54.329451 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:06:54.329451 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:06:54.329451 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:06:54.329451 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:06:54.329451 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:06:54.329451 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:06:54.329451 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:06:54.329451 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jun 29 11:06:54.329451 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:06:54.329451 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:06:54.329451 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:06:54.329451 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:06:54.329451 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:06:54.329451 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:06:54.329451 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:06:54.329451 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:06:54.331482 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 11:06:54.331482 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 11:06:54.331482 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 11:06:54.331482 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jun 29 11:06:54.331482 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 311/615] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jun 29 11:06:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:06:54.337579 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c26a042-d650-4ee0-8c3d-f3cc08e426a8 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:54.403967 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:54.415286 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:54.417238 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0f1576-bd40-472c-84e2-f736036ce950 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:54.451596 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c26a042-d650-4ee0-8c3d-f3cc08e426a8 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:54.455072 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f76784-6021-4214-b9fe-60f9da64a689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:54.456105 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f76784-6021-4214-b9fe-60f9da64a689 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:54.456599 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f76784-6021-4214-b9fe-60f9da64a689 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:54.457074 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f76784-6021-4214-b9fe-60f9da64a689 None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:54.458117 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f76784-6021-4214-b9fe-60f9da64a689 None admin] RBAC: Adding query filter params (name=service) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:06:54.458626 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f76784-6021-4214-b9fe-60f9da64a689 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:54.460240 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f76784-6021-4214-b9fe-60f9da64a689 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:54.469539 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 305/616] 10.222.0.44 () {56 vars in 1103 bytes} [Wed Jun 29 11:06:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 11:06:54.475104 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f76784-6021-4214-b9fe-60f9da64a689 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:54.509144 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7d818f-d6f2-42a4-bfe7-31b80944f633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:54.509850 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7d818f-d6f2-42a4-bfe7-31b80944f633 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:54.510149 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7d818f-d6f2-42a4-bfe7-31b80944f633 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:54.510459 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7d818f-d6f2-42a4-bfe7-31b80944f633 None admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:54.511205 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7d818f-d6f2-42a4-bfe7-31b80944f633 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:06:54.511562 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7d818f-d6f2-42a4-bfe7-31b80944f633 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:54.512961 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7d818f-d6f2-42a4-bfe7-31b80944f633 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:54.538455 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 312/617] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Jun 29 11:06:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 11:06:54.544108 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7d818f-d6f2-42a4-bfe7-31b80944f633 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:54.583136 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f76784-6021-4214-b9fe-60f9da64a689 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:54.618720 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578b6558-eaf5-4d7f-89c8-4f1ab1e1a0fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:54.619885 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578b6558-eaf5-4d7f-89c8-4f1ab1e1a0fb None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:54.620394 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578b6558-eaf5-4d7f-89c8-4f1ab1e1a0fb None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:54.620868 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578b6558-eaf5-4d7f-89c8-4f1ab1e1a0fb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:54.632730 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578b6558-eaf5-4d7f-89c8-4f1ab1e1a0fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:54.634458 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578b6558-eaf5-4d7f-89c8-4f1ab1e1a0fb None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:54.652910 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-578b6558-eaf5-4d7f-89c8-4f1ab1e1a0fb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 11:06:54.652910 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:06:54.652910 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:06:54.652910 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:06:54.652910 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:06:54.652910 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:06:54.652910 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:06:54.652910 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:06:54.652910 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:06:54.652910 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:06:54.652910 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:06:54.652910 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:06:54.652910 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 11:06:54.652910 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 11:06:54.652910 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 11:06:54.653833 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 11:06:54.653833 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:06:54.653833 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:06:54.653833 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:06:54.653833 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:06:54.653833 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:06:54.653833 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:06:54.653833 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:06:54.653833 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jun 29 11:06:54.653833 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:06:54.653833 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:06:54.653833 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:06:54.653833 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:06:54.653833 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:06:54.653833 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:06:54.653833 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:06:54.653833 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:06:54.654754 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 11:06:54.654754 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 11:06:54.654754 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 11:06:54.654754 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:06:54.657075 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 306/618] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jun 29 11:06:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 11:06:54.662594 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578b6558-eaf5-4d7f-89c8-4f1ab1e1a0fb None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:54.665629 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:54.736564 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7d818f-d6f2-42a4-bfe7-31b80944f633 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:54.790992 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:54.798759 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578b6558-eaf5-4d7f-89c8-4f1ab1e1a0fb None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:54.813042 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b58fdbb-ebb2-45ac-8ec1-5116f37c66c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:54.814535 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b58fdbb-ebb2-45ac-8ec1-5116f37c66c1 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:54.815022 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b58fdbb-ebb2-45ac-8ec1-5116f37c66c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:54.815596 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b58fdbb-ebb2-45ac-8ec1-5116f37c66c1 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:54.830270 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b58fdbb-ebb2-45ac-8ec1-5116f37c66c1 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:54.832082 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b58fdbb-ebb2-45ac-8ec1-5116f37c66c1 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:54.847273 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-6b58fdbb-ebb2-45ac-8ec1-5116f37c66c1 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 29 11:06:54.847273 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:06:54.847273 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:06:54.847273 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:06:54.847273 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:06:54.847273 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:06:54.847273 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:06:54.847273 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:06:54.847273 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:06:54.847273 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:06:54.847273 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:06:54.847273 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:06:54.847273 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 11:06:54.847273 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 11:06:54.847273 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 11:06:54.847273 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 11:06:54.848841 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:06:54.848841 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:06:54.848841 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 11:06:54.848841 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 11:06:54.848841 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 11:06:54.848841 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 11:06:54.848841 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:06:54.848841 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:06:54.848841 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:06:54.848841 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:06:54.848841 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:06:54.848841 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jun 29 11:06:54.848841 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:06:54.848841 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:06:54.848841 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:06:54.848841 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:06:54.848841 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:06:54.848841 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:06:54.849803 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:06:54.849803 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:06:54.849803 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 11:06:54.849803 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 11:06:54.849803 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 11:06:54.849803 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 11:06:54.849803 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 11:06:54.849803 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 11:06:54.849803 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 29 11:06:54.849803 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jun 29 11:06:54.852466 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 313/619] 10.222.0.44 () {56 vars in 1096 bytes} [Wed Jun 29 11:06:54 2022] GET /identity/v3/users/swift => generated 83 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:06:54.857609 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b58fdbb-ebb2-45ac-8ec1-5116f37c66c1 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:54.884074 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621a2f65-2b8d-4546-acdc-48ec744c360d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:54.885413 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621a2f65-2b8d-4546-acdc-48ec744c360d None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:54.885897 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621a2f65-2b8d-4546-acdc-48ec744c360d None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:54.887339 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621a2f65-2b8d-4546-acdc-48ec744c360d None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:54.888300 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621a2f65-2b8d-4546-acdc-48ec744c360d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:06:54.888970 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621a2f65-2b8d-4546-acdc-48ec744c360d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:54.891627 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621a2f65-2b8d-4546-acdc-48ec744c360d None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:54.914608 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 307/620] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Jun 29 11:06:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 11:06:54.925345 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621a2f65-2b8d-4546-acdc-48ec744c360d None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:55.005344 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:55.017232 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b58fdbb-ebb2-45ac-8ec1-5116f37c66c1 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:55.044146 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621a2f65-2b8d-4546-acdc-48ec744c360d None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:55.061105 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dc6ab2-1073-4fa0-a9d1-1633494e8ee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:55.062011 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dc6ab2-1073-4fa0-a9d1-1633494e8ee4 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:55.062669 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dc6ab2-1073-4fa0-a9d1-1633494e8ee4 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:55.062952 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dc6ab2-1073-4fa0-a9d1-1633494e8ee4 None admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:55.063792 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0dc6ab2-1073-4fa0-a9d1-1633494e8ee4 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:06:55.064141 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0dc6ab2-1073-4fa0-a9d1-1633494e8ee4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:55.066107 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0dc6ab2-1073-4fa0-a9d1-1633494e8ee4 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:55.080886 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da293e6-210a-4f13-a9b4-e350dca3e4d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:55.081752 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da293e6-210a-4f13-a9b4-e350dca3e4d3 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:55.082108 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da293e6-210a-4f13-a9b4-e350dca3e4d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:55.082460 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da293e6-210a-4f13-a9b4-e350dca3e4d3 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:55.088316 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 314/621] 10.222.0.44 () {56 vars in 1135 bytes} [Wed Jun 29 11:06:54 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 11:06:55.089596 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9a6258-6073-449c-99db-44b4d9344172 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:55.089930 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9a6258-6073-449c-99db-44b4d9344172 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:55.090214 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9a6258-6073-449c-99db-44b4d9344172 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:55.091819 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da293e6-210a-4f13-a9b4-e350dca3e4d3 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:55.092008 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 315/622] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:06:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:06:55.094244 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9a6258-6073-449c-99db-44b4d9344172 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:55.097346 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da293e6-210a-4f13-a9b4-e350dca3e4d3 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:55.116872 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-4da293e6-210a-4f13-a9b4-e350dca3e4d3 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 29 11:06:55.116872 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:06:55.116872 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:06:55.116872 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:06:55.116872 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:06:55.116872 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:06:55.116872 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:06:55.116872 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:06:55.116872 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:06:55.116872 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:06:55.116872 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:06:55.116872 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:06:55.116872 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 11:06:55.116872 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 11:06:55.116872 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 11:06:55.116872 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 11:06:55.117832 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:06:55.117832 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:06:55.117832 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 11:06:55.117832 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 11:06:55.117832 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 11:06:55.117832 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 11:06:55.117832 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:06:55.117832 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:06:55.117832 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:06:55.117832 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:06:55.117832 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:06:55.117832 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jun 29 11:06:55.117832 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:06:55.117832 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:06:55.117832 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:06:55.117832 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:06:55.117832 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:06:55.117832 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:06:55.118787 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:06:55.118787 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:06:55.118787 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 11:06:55.118787 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 11:06:55.118787 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 11:06:55.118787 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 11:06:55.118787 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 11:06:55.118787 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 11:06:55.118787 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 29 11:06:55.118787 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:06:55.121416 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 308/623] 10.222.0.44 () {56 vars in 1093 bytes} [Wed Jun 29 11:06:54 2022] GET /identity/v3/users/nova => generated 82 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 11:06:55.122735 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da216a41-9bf5-4044-86e6-33138b9a7f66 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:55.123210 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da216a41-9bf5-4044-86e6-33138b9a7f66 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:55.123655 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da216a41-9bf5-4044-86e6-33138b9a7f66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:55.141132 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-da216a41-9bf5-4044-86e6-33138b9a7f66 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:06:55.142595 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-da216a41-9bf5-4044-86e6-33138b9a7f66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:06:55.153654 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:55.216503 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 309/624] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:06:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 11:06:55.217670 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da216a41-9bf5-4044-86e6-33138b9a7f66 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:55.237187 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9a6258-6073-449c-99db-44b4d9344172 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:55.277429 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42117219-ae35-4384-bfa4-9ca5b7dab4ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:55.278633 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42117219-ae35-4384-bfa4-9ca5b7dab4ce None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:55.279111 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42117219-ae35-4384-bfa4-9ca5b7dab4ce None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:55.279579 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42117219-ae35-4384-bfa4-9ca5b7dab4ce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:55.295855 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42117219-ae35-4384-bfa4-9ca5b7dab4ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:55.295855 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42117219-ae35-4384-bfa4-9ca5b7dab4ce None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:55.296097 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:55.321394 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-42117219-ae35-4384-bfa4-9ca5b7dab4ce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 11:06:55.321394 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:06:55.321394 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:06:55.321394 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:06:55.321394 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:06:55.321394 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:06:55.321394 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:06:55.321394 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:06:55.321394 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:06:55.321394 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:06:55.321394 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:06:55.321394 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:06:55.321394 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 11:06:55.321394 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 11:06:55.321394 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 11:06:55.322736 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 11:06:55.322736 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:06:55.322736 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:06:55.322736 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:06:55.322736 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:06:55.322736 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:06:55.322736 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:06:55.322736 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:06:55.322736 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jun 29 11:06:55.322736 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:06:55.322736 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:06:55.322736 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:06:55.322736 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:06:55.322736 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:06:55.322736 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:06:55.322736 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:06:55.322736 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:06:55.323616 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 11:06:55.323616 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 11:06:55.323616 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 11:06:55.323616 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jun 29 11:06:55.323866 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 316/625] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jun 29 11:06:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:06:55.328824 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42117219-ae35-4384-bfa4-9ca5b7dab4ce None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:55.357864 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da216a41-9bf5-4044-86e6-33138b9a7f66 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:55.401885 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f203eac6-cfe4-4706-9dbc-8d3e81873ddd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:55.402666 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f203eac6-cfe4-4706-9dbc-8d3e81873ddd None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:55.402951 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f203eac6-cfe4-4706-9dbc-8d3e81873ddd None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:55.403225 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f203eac6-cfe4-4706-9dbc-8d3e81873ddd None admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:55.404101 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f203eac6-cfe4-4706-9dbc-8d3e81873ddd None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:06:55.404411 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f203eac6-cfe4-4706-9dbc-8d3e81873ddd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:55.406069 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f203eac6-cfe4-4706-9dbc-8d3e81873ddd None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:55.425995 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 310/626] 10.222.0.44 () {56 vars in 1133 bytes} [Wed Jun 29 11:06:55 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:55.427524 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f203eac6-cfe4-4706-9dbc-8d3e81873ddd None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:55.435938 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:55.439745 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42117219-ae35-4384-bfa4-9ca5b7dab4ce None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:55.474919 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c10a943-390b-4472-8df6-078013c72ccf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:55.475808 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c10a943-390b-4472-8df6-078013c72ccf None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:55.476204 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c10a943-390b-4472-8df6-078013c72ccf None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:55.476559 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c10a943-390b-4472-8df6-078013c72ccf None admin] PATH_INFO: `/v3/roles/service` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:55.487657 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c10a943-390b-4472-8df6-078013c72ccf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 11:06:55.496374 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-9c10a943-390b-4472-8df6-078013c72ccf None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 29 11:06:55.496374 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:06:55.496374 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:06:55.496374 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:06:55.496374 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:06:55.496374 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:06:55.496374 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:06:55.496374 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:06:55.496374 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:06:55.496374 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:06:55.496374 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:06:55.496374 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:06:55.496374 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 11:06:55.496374 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 11:06:55.496374 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 11:06:55.496374 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 11:06:55.497524 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 11:06:55.497524 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 11:06:55.497524 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 11:06:55.497524 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 11:06:55.497524 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:06:55.497524 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:06:55.497524 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:06:55.497524 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:06:55.497524 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:06:55.497524 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:06:55.497524 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:06:55.497524 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jun 29 11:06:55.497524 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:06:55.497524 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:06:55.497524 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:06:55.497524 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:06:55.499129 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:06:55.499129 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:06:55.499129 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:06:55.499129 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:06:55.499129 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 11:06:55.499129 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 11:06:55.499129 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 11:06:55.499129 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 11:06:55.499129 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 11:06:55.499129 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 11:06:55.499129 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 29 11:06:55.499129 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jun 29 11:06:55.504674 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 317/627] 10.222.0.44 () {56 vars in 1102 bytes} [Wed Jun 29 11:06:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:06:55.505622 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c10a943-390b-4472-8df6-078013c72ccf None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:55.534356 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f203eac6-cfe4-4706-9dbc-8d3e81873ddd None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:55.546276 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:55.571463 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6995fd46-773a-4dd0-8e12-8ba2de5676fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:55.572433 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6995fd46-773a-4dd0-8e12-8ba2de5676fb None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:55.572703 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6995fd46-773a-4dd0-8e12-8ba2de5676fb None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:55.573066 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6995fd46-773a-4dd0-8e12-8ba2de5676fb None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:55.573789 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6995fd46-773a-4dd0-8e12-8ba2de5676fb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:06:55.574199 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6995fd46-773a-4dd0-8e12-8ba2de5676fb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:55.575566 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6995fd46-773a-4dd0-8e12-8ba2de5676fb None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:55.588874 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 311/628] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Jun 29 11:06:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:55.589754 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6995fd46-773a-4dd0-8e12-8ba2de5676fb None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:55.610986 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c10a943-390b-4472-8df6-078013c72ccf None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:55.645585 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f47e09-35dd-478c-b5d5-46fae844ecb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:55.647299 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f47e09-35dd-478c-b5d5-46fae844ecb3 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:55.647299 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f47e09-35dd-478c-b5d5-46fae844ecb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:55.647299 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f47e09-35dd-478c-b5d5-46fae844ecb3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:55.675397 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:55.697935 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6995fd46-773a-4dd0-8e12-8ba2de5676fb None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:55.718764 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f47e09-35dd-478c-b5d5-46fae844ecb3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:55.720396 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f47e09-35dd-478c-b5d5-46fae844ecb3 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:55.733426 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792f612d-3c06-4ddb-a9bb-23da9bb08eef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:55.733688 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792f612d-3c06-4ddb-a9bb-23da9bb08eef None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:55.733999 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792f612d-3c06-4ddb-a9bb-23da9bb08eef None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:55.734239 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792f612d-3c06-4ddb-a9bb-23da9bb08eef None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:55.734976 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792f612d-3c06-4ddb-a9bb-23da9bb08eef None admin] RBAC: Adding query filter params (name=service) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:06:55.735297 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792f612d-3c06-4ddb-a9bb-23da9bb08eef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:55.738073 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792f612d-3c06-4ddb-a9bb-23da9bb08eef None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:55.745879 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 312/629] 10.222.0.44 () {56 vars in 1103 bytes} [Wed Jun 29 11:06:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:55.746743 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792f612d-3c06-4ddb-a9bb-23da9bb08eef None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:55.778828 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:55.840000 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 318/630] 10.222.0.44 () {58 vars in 1282 bytes} [Wed Jun 29 11:06:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 29 11:06:55.840852 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f47e09-35dd-478c-b5d5-46fae844ecb3 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:55.860968 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792f612d-3c06-4ddb-a9bb-23da9bb08eef None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:55.898389 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:55.900650 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1967c1ab-cee5-43bf-81a6-fdb2f2fae254 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:55.901986 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1967c1ab-cee5-43bf-81a6-fdb2f2fae254 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:55.902269 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1967c1ab-cee5-43bf-81a6-fdb2f2fae254 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:55.902544 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1967c1ab-cee5-43bf-81a6-fdb2f2fae254 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:55.909432 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1967c1ab-cee5-43bf-81a6-fdb2f2fae254 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:55.911559 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1967c1ab-cee5-43bf-81a6-fdb2f2fae254 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:55.925556 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-1967c1ab-cee5-43bf-81a6-fdb2f2fae254 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 11:06:55.925556 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:06:55.925556 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:06:55.925556 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:06:55.925556 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:06:55.925556 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:06:55.925556 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:06:55.925556 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:06:55.925556 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:06:55.925556 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:06:55.925556 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:06:55.925556 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:06:55.925556 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 11:06:55.925556 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 11:06:55.925556 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 11:06:55.926483 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 11:06:55.926483 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:06:55.926483 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:06:55.926483 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:06:55.926483 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:06:55.926483 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:06:55.926483 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:06:55.926483 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:06:55.926483 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jun 29 11:06:55.926483 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:06:55.926483 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:06:55.926483 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:06:55.926483 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:06:55.926483 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:06:55.926483 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:06:55.926483 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:06:55.926483 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:06:55.927364 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 11:06:55.927364 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 11:06:55.927364 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 11:06:55.927364 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 11:06:55.927364 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 11:06:55.927364 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 11:06:55.927364 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 11:06:55.927364 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:06:55.927364 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 313/631] 10.222.0.44 () {56 vars in 1111 bytes} [Wed Jun 29 11:06:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:06:55.928389 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae564e73-cbbd-4d94-9128-84f0a9373ba4 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:55.928389 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae564e73-cbbd-4d94-9128-84f0a9373ba4 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:55.928604 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae564e73-cbbd-4d94-9128-84f0a9373ba4 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:55.932020 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 314/632] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:06:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:06:55.932020 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae564e73-cbbd-4d94-9128-84f0a9373ba4 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:55.951985 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f47e09-35dd-478c-b5d5-46fae844ecb3 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:55.987248 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb3ec58-62a1-4bc0-a7ae-adcff52dee23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:55.988001 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb3ec58-62a1-4bc0-a7ae-adcff52dee23 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:55.988284 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb3ec58-62a1-4bc0-a7ae-adcff52dee23 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:55.988561 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb3ec58-62a1-4bc0-a7ae-adcff52dee23 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:55.999332 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb3ec58-62a1-4bc0-a7ae-adcff52dee23 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:56.000813 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb3ec58-62a1-4bc0-a7ae-adcff52dee23 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:56.018701 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-dfb3ec58-62a1-4bc0-a7ae-adcff52dee23 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 11:06:56.018701 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:06:56.018701 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:06:56.018701 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:06:56.018701 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:06:56.018701 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:06:56.018701 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:06:56.018701 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:06:56.018701 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:06:56.018701 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:06:56.018701 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:06:56.018701 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:06:56.018701 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 11:06:56.018701 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 11:06:56.018701 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 11:06:56.019629 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 11:06:56.019629 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:06:56.019629 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:06:56.019629 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:06:56.019629 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:06:56.019629 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:06:56.019629 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:06:56.019629 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:06:56.019629 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jun 29 11:06:56.019629 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:06:56.019629 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:06:56.019629 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:06:56.019629 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:06:56.019629 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:06:56.019629 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:06:56.019629 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:06:56.019629 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:06:56.021052 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 11:06:56.021052 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 11:06:56.021052 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 11:06:56.021052 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jun 29 11:06:56.021052 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 319/633] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jun 29 11:06:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:06:56.021894 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb3ec58-62a1-4bc0-a7ae-adcff52dee23 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:56.041583 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:56.052524 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae564e73-cbbd-4d94-9128-84f0a9373ba4 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:56.089841 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0aa4383-a2bf-482e-bb8e-ea8d5299e05b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:56.090631 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0aa4383-a2bf-482e-bb8e-ea8d5299e05b None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:56.090916 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0aa4383-a2bf-482e-bb8e-ea8d5299e05b None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:56.091194 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0aa4383-a2bf-482e-bb8e-ea8d5299e05b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:56.103133 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0aa4383-a2bf-482e-bb8e-ea8d5299e05b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:56.104721 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0aa4383-a2bf-482e-bb8e-ea8d5299e05b None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:56.122423 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb3ec58-62a1-4bc0-a7ae-adcff52dee23 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:56.123808 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-c0aa4383-a2bf-482e-bb8e-ea8d5299e05b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 11:06:56.123808 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:06:56.123808 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:06:56.123808 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:06:56.123808 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:06:56.123808 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:06:56.123808 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:06:56.123808 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:06:56.123808 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:06:56.123808 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:06:56.123808 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:06:56.123808 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:06:56.123808 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 11:06:56.123808 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 11:06:56.123808 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 11:06:56.124707 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 11:06:56.124707 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:06:56.124707 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:06:56.124707 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:06:56.124707 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:06:56.124707 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:06:56.124707 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:06:56.124707 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:06:56.124707 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jun 29 11:06:56.124707 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:06:56.124707 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:06:56.124707 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:06:56.124707 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:06:56.124707 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:06:56.124707 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:06:56.124707 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:06:56.124707 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:06:56.125620 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 11:06:56.125620 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 11:06:56.125620 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 11:06:56.125620 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:06:56.125620 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 315/634] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jun 29 11:06:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:06:56.126632 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571dbc23-7cb5-4ef8-9ff1-60dea3f58b44 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:56.126919 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571dbc23-7cb5-4ef8-9ff1-60dea3f58b44 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:56.127193 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571dbc23-7cb5-4ef8-9ff1-60dea3f58b44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:56.137631 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:56.147198 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-571dbc23-7cb5-4ef8-9ff1-60dea3f58b44 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:06:56.148415 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-571dbc23-7cb5-4ef8-9ff1-60dea3f58b44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:06:56.162252 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d4047e-9cba-4099-9470-e906a20fec4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:56.163013 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d4047e-9cba-4099-9470-e906a20fec4f None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:56.163298 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d4047e-9cba-4099-9470-e906a20fec4f None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:56.163580 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d4047e-9cba-4099-9470-e906a20fec4f None admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:56.164398 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d4047e-9cba-4099-9470-e906a20fec4f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:06:56.164711 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d4047e-9cba-4099-9470-e906a20fec4f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:56.166390 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d4047e-9cba-4099-9470-e906a20fec4f None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:56.184083 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 320/635] 10.222.0.44 () {56 vars in 1148 bytes} [Wed Jun 29 11:06:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:56.184840 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d4047e-9cba-4099-9470-e906a20fec4f None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:56.223856 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 316/636] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:06:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 11:06:56.223856 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571dbc23-7cb5-4ef8-9ff1-60dea3f58b44 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:56.300680 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:56.303751 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d4047e-9cba-4099-9470-e906a20fec4f None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:56.326036 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571dbc23-7cb5-4ef8-9ff1-60dea3f58b44 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:56.338281 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a38c84-8aa2-4980-9091-fff33a6c47d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:56.339136 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a38c84-8aa2-4980-9091-fff33a6c47d6 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:56.339417 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a38c84-8aa2-4980-9091-fff33a6c47d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:56.339701 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a38c84-8aa2-4980-9091-fff33a6c47d6 None admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:56.340459 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a38c84-8aa2-4980-9091-fff33a6c47d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:06:56.340774 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a38c84-8aa2-4980-9091-fff33a6c47d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:56.342132 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a38c84-8aa2-4980-9091-fff33a6c47d6 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:56.353708 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 321/637] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Jun 29 11:06:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:56.354465 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a38c84-8aa2-4980-9091-fff33a6c47d6 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:56.360579 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee94dcb-8ea2-445b-9643-4d4850c1336e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:56.361269 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee94dcb-8ea2-445b-9643-4d4850c1336e None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:56.361546 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee94dcb-8ea2-445b-9643-4d4850c1336e None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:56.361819 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee94dcb-8ea2-445b-9643-4d4850c1336e None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:56.362531 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee94dcb-8ea2-445b-9643-4d4850c1336e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:06:56.362834 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee94dcb-8ea2-445b-9643-4d4850c1336e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:56.364251 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee94dcb-8ea2-445b-9643-4d4850c1336e None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:56.377066 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 317/638] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Jun 29 11:06:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:56.378462 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee94dcb-8ea2-445b-9643-4d4850c1336e None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:56.387526 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:56.455820 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a38c84-8aa2-4980-9091-fff33a6c47d6 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:56.482003 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee94dcb-8ea2-445b-9643-4d4850c1336e None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:56.490112 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514e346c-1bb0-46fb-b44e-ca10b30f4ddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:56.490994 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514e346c-1bb0-46fb-b44e-ca10b30f4ddf None admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:56.491274 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514e346c-1bb0-46fb-b44e-ca10b30f4ddf None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:56.491545 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514e346c-1bb0-46fb-b44e-ca10b30f4ddf None admin] PATH_INFO: `/v3/projects/ebe5dffa4bb8440286687daefb6c98f4/users/e3b38493ba904759badc800e396befda/roles/a8212be3426741cbb1b38b1646c66405` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:56.505989 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514e346c-1bb0-46fb-b44e-ca10b30f4ddf None admin] RBAC: Authorizing `identity:create_grant(project_id=ebe5dffa4bb8440286687daefb6c98f4, user_id=e3b38493ba904759badc800e396befda, role_id=a8212be3426741cbb1b38b1646c66405)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:56.507408 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514e346c-1bb0-46fb-b44e-ca10b30f4ddf None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:56.515740 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1195fad-169d-454f-862c-e85d96f581b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:56.516558 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1195fad-169d-454f-862c-e85d96f581b1 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:56.516845 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1195fad-169d-454f-862c-e85d96f581b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:56.517124 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1195fad-169d-454f-862c-e85d96f581b1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:56.518952 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:56.527970 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1195fad-169d-454f-862c-e85d96f581b1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 11:06:56.535732 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-d1195fad-169d-454f-862c-e85d96f581b1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 29 11:06:56.535732 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:06:56.535732 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:06:56.535732 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:06:56.535732 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:06:56.535732 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:06:56.535732 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:06:56.535732 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:06:56.535732 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:06:56.535732 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:06:56.535732 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:06:56.535732 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:06:56.535732 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 11:06:56.535732 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 11:06:56.535732 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 11:06:56.535732 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 11:06:56.536858 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 11:06:56.536858 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 11:06:56.536858 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 11:06:56.536858 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 11:06:56.536858 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:06:56.536858 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:06:56.536858 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:06:56.536858 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:06:56.536858 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:06:56.536858 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:06:56.536858 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:06:56.536858 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jun 29 11:06:56.536858 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:06:56.536858 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:06:56.536858 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:06:56.536858 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:06:56.537894 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:06:56.537894 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:06:56.537894 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:06:56.537894 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:06:56.537894 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 11:06:56.537894 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 11:06:56.537894 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 11:06:56.537894 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 11:06:56.537894 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 11:06:56.537894 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 11:06:56.537894 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 29 11:06:56.537894 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:06:56.537894 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 318/639] 10.222.0.44 () {56 vars in 1102 bytes} [Wed Jun 29 11:06:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:06:56.538811 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09096f37-ecb0-4861-b529-30258f84f82b None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:56.538811 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09096f37-ecb0-4861-b529-30258f84f82b None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:56.538811 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09096f37-ecb0-4861-b529-30258f84f82b None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:56.540026 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 319/640] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:06:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:06:56.540711 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09096f37-ecb0-4861-b529-30258f84f82b None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:56.584404 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 322/641] 10.222.0.44 () {58 vars in 1439 bytes} [Wed Jun 29 11:06:56 2022] PUT /identity/v3/projects/ebe5dffa4bb8440286687daefb6c98f4/users/e3b38493ba904759badc800e396befda/roles/a8212be3426741cbb1b38b1646c66405 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:06:56.585159 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514e346c-1bb0-46fb-b44e-ca10b30f4ddf None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:56.649659 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09096f37-ecb0-4861-b529-30258f84f82b None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:56.687876 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6325477f-2982-4917-9b99-2cb98c7d3e01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:56.689019 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6325477f-2982-4917-9b99-2cb98c7d3e01 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:56.689307 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6325477f-2982-4917-9b99-2cb98c7d3e01 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:56.689592 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6325477f-2982-4917-9b99-2cb98c7d3e01 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:56.693806 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:56.698552 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6325477f-2982-4917-9b99-2cb98c7d3e01 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:56.700112 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6325477f-2982-4917-9b99-2cb98c7d3e01 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:56.715674 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514e346c-1bb0-46fb-b44e-ca10b30f4ddf None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:56.716271 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-6325477f-2982-4917-9b99-2cb98c7d3e01 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 29 11:06:56.716271 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:06:56.716271 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:06:56.716271 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:06:56.716271 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:06:56.716271 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:06:56.716271 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:06:56.716271 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:06:56.716271 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:06:56.716271 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:06:56.716271 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:06:56.716271 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:06:56.716271 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 11:06:56.716271 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 11:06:56.716271 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 11:06:56.716271 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 11:06:56.717264 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:06:56.717264 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:06:56.717264 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 11:06:56.717264 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 11:06:56.717264 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 11:06:56.717264 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 11:06:56.717264 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:06:56.717264 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:06:56.717264 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:06:56.717264 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:06:56.717264 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:06:56.717264 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jun 29 11:06:56.717264 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:06:56.717264 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:06:56.717264 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:06:56.717264 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:06:56.717264 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:06:56.717264 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:06:56.718226 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:06:56.718226 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:06:56.718226 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 11:06:56.718226 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 11:06:56.718226 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 11:06:56.718226 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 11:06:56.718226 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 11:06:56.718226 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 11:06:56.718226 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 29 11:06:56.718226 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:06:56.725008 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 320/642] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Jun 29 11:06:56 2022] GET /identity/v3/users/glance => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:06:56.726242 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6325477f-2982-4917-9b99-2cb98c7d3e01 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:56.756899 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec55afcf-a2b1-43d8-a2e0-5be48cda0015 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:56.757709 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec55afcf-a2b1-43d8-a2e0-5be48cda0015 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:56.757978 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec55afcf-a2b1-43d8-a2e0-5be48cda0015 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:56.758257 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec55afcf-a2b1-43d8-a2e0-5be48cda0015 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:56.765029 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec55afcf-a2b1-43d8-a2e0-5be48cda0015 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:56.766479 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec55afcf-a2b1-43d8-a2e0-5be48cda0015 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:56.781245 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-ec55afcf-a2b1-43d8-a2e0-5be48cda0015 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 11:06:56.781245 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:06:56.781245 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:06:56.781245 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:06:56.781245 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:06:56.781245 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:06:56.781245 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:06:56.781245 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:06:56.781245 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:06:56.781245 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:06:56.781245 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:06:56.781245 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:06:56.781245 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 11:06:56.781245 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 11:06:56.781245 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 11:06:56.782191 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 11:06:56.782191 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:06:56.782191 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:06:56.782191 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:06:56.782191 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:06:56.782191 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:06:56.782191 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:06:56.782191 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:06:56.782191 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jun 29 11:06:56.782191 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:06:56.782191 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:06:56.782191 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:06:56.782191 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:06:56.782191 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:06:56.782191 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:06:56.782191 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:06:56.782191 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:06:56.783160 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 11:06:56.783160 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 11:06:56.783160 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 11:06:56.783160 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 11:06:56.783160 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 11:06:56.783160 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 11:06:56.783160 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 11:06:56.783160 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jun 29 11:06:56.783160 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 323/643] 10.222.0.44 () {56 vars in 1111 bytes} [Wed Jun 29 11:06:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:06:56.784145 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc106c7-5a3b-48d6-94a8-bc0dcf464f1e None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:56.784145 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc106c7-5a3b-48d6-94a8-bc0dcf464f1e None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:56.784145 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc106c7-5a3b-48d6-94a8-bc0dcf464f1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:56.794502 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:56.801007 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-5fc106c7-5a3b-48d6-94a8-bc0dcf464f1e None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:06:56.802749 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-5fc106c7-5a3b-48d6-94a8-bc0dcf464f1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:06:56.841836 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6325477f-2982-4917-9b99-2cb98c7d3e01 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:56.870139 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 324/644] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:06:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 11:06:56.870982 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc106c7-5a3b-48d6-94a8-bc0dcf464f1e None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:56.883273 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0cff1e-8e29-4229-8cbb-4000ae10896a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:56.884104 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0cff1e-8e29-4229-8cbb-4000ae10896a None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:56.884489 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0cff1e-8e29-4229-8cbb-4000ae10896a None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:56.884865 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0cff1e-8e29-4229-8cbb-4000ae10896a None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:56.885718 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0cff1e-8e29-4229-8cbb-4000ae10896a None admin] RBAC: Adding query filter params (name=service) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:06:56.886180 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0cff1e-8e29-4229-8cbb-4000ae10896a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:56.887643 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0cff1e-8e29-4229-8cbb-4000ae10896a None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:56.899564 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 321/645] 10.222.0.44 () {56 vars in 1103 bytes} [Wed Jun 29 11:06:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:56.900623 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0cff1e-8e29-4229-8cbb-4000ae10896a None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:56.961496 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:56.997965 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc106c7-5a3b-48d6-94a8-bc0dcf464f1e None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:57.016710 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0cff1e-8e29-4229-8cbb-4000ae10896a None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:57.033539 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:57.035196 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6566c619-27e4-42c4-9267-23802425f9eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:57.036417 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6566c619-27e4-42c4-9267-23802425f9eb None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:57.036709 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6566c619-27e4-42c4-9267-23802425f9eb None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:57.036985 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6566c619-27e4-42c4-9267-23802425f9eb None admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:57.038256 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6566c619-27e4-42c4-9267-23802425f9eb None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:06:57.038572 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6566c619-27e4-42c4-9267-23802425f9eb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:57.040035 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6566c619-27e4-42c4-9267-23802425f9eb None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:57.053954 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29756fc0-6abe-4ee9-b073-3a001e36f924 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:57.055144 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29756fc0-6abe-4ee9-b073-3a001e36f924 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:57.055744 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29756fc0-6abe-4ee9-b073-3a001e36f924 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:57.056538 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29756fc0-6abe-4ee9-b073-3a001e36f924 None admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:57.057332 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29756fc0-6abe-4ee9-b073-3a001e36f924 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:06:57.057648 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29756fc0-6abe-4ee9-b073-3a001e36f924 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:57.062526 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 325/646] 10.222.0.44 () {56 vars in 1137 bytes} [Wed Jun 29 11:06:56 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:57.062526 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6566c619-27e4-42c4-9267-23802425f9eb None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:57.062716 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29756fc0-6abe-4ee9-b073-3a001e36f924 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:57.078581 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 322/647] 10.222.0.44 () {56 vars in 1148 bytes} [Wed Jun 29 11:06:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:57.078581 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29756fc0-6abe-4ee9-b073-3a001e36f924 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:57.163331 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6566c619-27e4-42c4-9267-23802425f9eb None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:57.185220 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29756fc0-6abe-4ee9-b073-3a001e36f924 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:57.189250 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:57.198220 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd1ab80-cc9f-458c-800f-8bb2616a6efd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:57.202535 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd1ab80-cc9f-458c-800f-8bb2616a6efd None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:57.202535 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd1ab80-cc9f-458c-800f-8bb2616a6efd None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:57.202535 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd1ab80-cc9f-458c-800f-8bb2616a6efd None admin] PATH_INFO: `/v3/roles/service` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:57.210554 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd1ab80-cc9f-458c-800f-8bb2616a6efd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 11:06:57.218389 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-3dd1ab80-cc9f-458c-800f-8bb2616a6efd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 29 11:06:57.218389 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:06:57.218389 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:06:57.218389 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:06:57.218389 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:06:57.218389 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:06:57.218389 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:06:57.218389 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:06:57.218389 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:06:57.218389 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:06:57.218389 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:06:57.218389 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:06:57.218389 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 11:06:57.218389 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 11:06:57.218389 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 11:06:57.218389 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 11:06:57.219491 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 11:06:57.219491 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 11:06:57.219491 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 11:06:57.219491 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 11:06:57.219491 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:06:57.219491 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:06:57.219491 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:06:57.219491 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:06:57.219491 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:06:57.219491 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:06:57.219491 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:06:57.219491 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jun 29 11:06:57.219491 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:06:57.219491 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:06:57.219491 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:06:57.219491 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:06:57.220570 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:06:57.220570 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:06:57.220570 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:06:57.220570 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:06:57.220570 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 11:06:57.220570 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 11:06:57.220570 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 11:06:57.220570 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 11:06:57.220570 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 11:06:57.220570 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 11:06:57.220570 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 29 11:06:57.220570 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jun 29 11:06:57.220570 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 326/648] 10.222.0.44 () {56 vars in 1102 bytes} [Wed Jun 29 11:06:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:06:57.221271 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd1ab80-cc9f-458c-800f-8bb2616a6efd None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:57.228147 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9aa4b4d-0451-433b-acc7-ac5323955865 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:57.228952 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9aa4b4d-0451-433b-acc7-ac5323955865 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:57.229241 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9aa4b4d-0451-433b-acc7-ac5323955865 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:57.229521 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9aa4b4d-0451-433b-acc7-ac5323955865 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:57.240852 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9aa4b4d-0451-433b-acc7-ac5323955865 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:57.242297 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9aa4b4d-0451-433b-acc7-ac5323955865 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:57.260384 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-d9aa4b4d-0451-433b-acc7-ac5323955865 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 11:06:57.260384 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:06:57.260384 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:06:57.260384 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:06:57.260384 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:06:57.260384 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:06:57.260384 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:06:57.260384 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:06:57.260384 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:06:57.260384 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:06:57.260384 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:06:57.260384 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:06:57.260384 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 11:06:57.260384 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 11:06:57.260384 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 11:06:57.262210 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 11:06:57.262210 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:06:57.262210 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:06:57.262210 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:06:57.262210 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:06:57.262210 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:06:57.262210 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:06:57.262210 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:06:57.262210 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jun 29 11:06:57.262210 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:06:57.262210 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:06:57.262210 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:06:57.262210 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:06:57.262210 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:06:57.262210 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:06:57.262210 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:06:57.262210 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:06:57.263143 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 11:06:57.263143 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 11:06:57.263143 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 11:06:57.263143 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:06:57.263143 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 323/649] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jun 29 11:06:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:06:57.263143 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9aa4b4d-0451-433b-acc7-ac5323955865 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:57.297825 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:57.347990 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd1ab80-cc9f-458c-800f-8bb2616a6efd None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:57.380261 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9aa4b4d-0451-433b-acc7-ac5323955865 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:57.382490 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083e0784-dc5a-407a-bcb6-ef4777f9dd4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:57.383129 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083e0784-dc5a-407a-bcb6-ef4777f9dd4b None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:57.383411 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083e0784-dc5a-407a-bcb6-ef4777f9dd4b None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:57.383687 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083e0784-dc5a-407a-bcb6-ef4777f9dd4b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:57.419384 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153538e2-f216-4c44-9366-1067c8cee833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:57.420709 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153538e2-f216-4c44-9366-1067c8cee833 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:57.420993 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153538e2-f216-4c44-9366-1067c8cee833 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:57.421261 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153538e2-f216-4c44-9366-1067c8cee833 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:57.422094 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153538e2-f216-4c44-9366-1067c8cee833 None admin] RBAC: Adding query filter params (role.id=a8212be3426741cbb1b38b1646c66405, scope.project.id=ebe5dffa4bb8440286687daefb6c98f4, user.id=796aeeff16df4739bddec1f6dee61bfe) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:06:57.422404 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153538e2-f216-4c44-9366-1067c8cee833 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:57.423851 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153538e2-f216-4c44-9366-1067c8cee833 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:57.431586 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 324/650] 10.222.0.44 () {56 vars in 1374 bytes} [Wed Jun 29 11:06:57 2022] GET /identity/v3/role_assignments?user.id=796aeeff16df4739bddec1f6dee61bfe&scope.project.id=ebe5dffa4bb8440286687daefb6c98f4&role.id=a8212be3426741cbb1b38b1646c66405 => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:57.432447 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153538e2-f216-4c44-9366-1067c8cee833 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:57.455819 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:57.469935 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083e0784-dc5a-407a-bcb6-ef4777f9dd4b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:57.479336 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083e0784-dc5a-407a-bcb6-ef4777f9dd4b None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:57.567628 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:57.580028 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153538e2-f216-4c44-9366-1067c8cee833 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:57.592131 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 327/651] 10.222.0.44 () {58 vars in 1282 bytes} [Wed Jun 29 11:06:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 29 11:06:57.593077 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083e0784-dc5a-407a-bcb6-ef4777f9dd4b None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:57.624655 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e2696e-b0cc-4ef7-8f2d-4fba7fdcec7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:57.626082 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e2696e-b0cc-4ef7-8f2d-4fba7fdcec7a None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:57.626433 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e2696e-b0cc-4ef7-8f2d-4fba7fdcec7a None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:57.626716 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e2696e-b0cc-4ef7-8f2d-4fba7fdcec7a None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:57.627458 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e2696e-b0cc-4ef7-8f2d-4fba7fdcec7a None admin] RBAC: Adding query filter params (name=service) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:06:57.627774 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e2696e-b0cc-4ef7-8f2d-4fba7fdcec7a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:57.629145 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e2696e-b0cc-4ef7-8f2d-4fba7fdcec7a None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:57.638709 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 325/652] 10.222.0.44 () {56 vars in 1103 bytes} [Wed Jun 29 11:06:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:57.639543 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e2696e-b0cc-4ef7-8f2d-4fba7fdcec7a None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:57.702314 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:57.720691 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083e0784-dc5a-407a-bcb6-ef4777f9dd4b None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:57.758108 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bab54c-0e4c-45b3-ac5e-b9ff2da6521c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:57.758773 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bab54c-0e4c-45b3-ac5e-b9ff2da6521c None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:57.759110 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bab54c-0e4c-45b3-ac5e-b9ff2da6521c None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:57.759386 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bab54c-0e4c-45b3-ac5e-b9ff2da6521c None admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:57.759558 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e2696e-b0cc-4ef7-8f2d-4fba7fdcec7a None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:57.760186 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bab54c-0e4c-45b3-ac5e-b9ff2da6521c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:06:57.760498 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bab54c-0e4c-45b3-ac5e-b9ff2da6521c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:57.762221 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bab54c-0e4c-45b3-ac5e-b9ff2da6521c None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:57.777936 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 328/653] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Jun 29 11:06:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:57.778660 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bab54c-0e4c-45b3-ac5e-b9ff2da6521c None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:57.795878 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39037199-9f29-41c4-b2e3-f4ed2a0ea351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:57.796734 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39037199-9f29-41c4-b2e3-f4ed2a0ea351 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:57.797017 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39037199-9f29-41c4-b2e3-f4ed2a0ea351 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:57.797290 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39037199-9f29-41c4-b2e3-f4ed2a0ea351 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:57.808483 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39037199-9f29-41c4-b2e3-f4ed2a0ea351 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:57.809928 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39037199-9f29-41c4-b2e3-f4ed2a0ea351 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:57.829021 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-39037199-9f29-41c4-b2e3-f4ed2a0ea351 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 11:06:57.829021 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:06:57.829021 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:06:57.829021 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:06:57.829021 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:06:57.829021 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:06:57.829021 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:06:57.829021 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:06:57.829021 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:06:57.829021 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:06:57.829021 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:06:57.829021 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:06:57.829021 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 11:06:57.829021 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 11:06:57.829021 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 11:06:57.829945 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 11:06:57.829945 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:06:57.829945 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:06:57.829945 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:06:57.829945 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:06:57.829945 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:06:57.829945 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:06:57.829945 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:06:57.829945 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jun 29 11:06:57.829945 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:06:57.829945 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:06:57.829945 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:06:57.829945 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:06:57.829945 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:06:57.829945 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:06:57.829945 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:06:57.829945 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:06:57.831495 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 11:06:57.831495 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 11:06:57.831495 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 11:06:57.831495 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:06:57.831495 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 326/654] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jun 29 11:06:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:06:57.831495 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39037199-9f29-41c4-b2e3-f4ed2a0ea351 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:57.844368 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:57.897219 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bab54c-0e4c-45b3-ac5e-b9ff2da6521c None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:57.943401 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39037199-9f29-41c4-b2e3-f4ed2a0ea351 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:57.950012 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb011af-6ae2-41d4-a744-557d5301350c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:57.950812 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb011af-6ae2-41d4-a744-557d5301350c None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:57.951080 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb011af-6ae2-41d4-a744-557d5301350c None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:57.951355 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb011af-6ae2-41d4-a744-557d5301350c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:57.965628 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb011af-6ae2-41d4-a744-557d5301350c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:57.967166 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb011af-6ae2-41d4-a744-557d5301350c None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:57.977776 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:57.988339 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-beb011af-6ae2-41d4-a744-557d5301350c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 11:06:57.988339 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:06:57.988339 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:06:57.988339 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:06:57.988339 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:06:57.988339 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:06:57.988339 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:06:57.988339 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:06:57.988339 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:06:57.988339 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:06:57.988339 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:06:57.988339 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:06:57.988339 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 11:06:57.988339 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 11:06:57.988339 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 11:06:57.989281 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 11:06:57.989281 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:06:57.989281 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:06:57.989281 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:06:57.989281 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:06:57.989281 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:06:57.989281 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:06:57.989281 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:06:57.989281 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jun 29 11:06:57.989281 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:06:57.989281 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:06:57.989281 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:06:57.989281 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:06:57.989281 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:06:57.989281 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:06:57.989281 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:06:57.989281 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:06:57.990665 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 11:06:57.990665 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 11:06:57.990665 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 11:06:57.990665 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jun 29 11:06:57.990665 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 329/655] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jun 29 11:06:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:06:57.990964 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb011af-6ae2-41d4-a744-557d5301350c None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:58.005542 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da41074-7e56-445a-a6a4-fb8b9aad2b6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:58.006777 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da41074-7e56-445a-a6a4-fb8b9aad2b6f None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:58.007644 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da41074-7e56-445a-a6a4-fb8b9aad2b6f None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:58.008290 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da41074-7e56-445a-a6a4-fb8b9aad2b6f None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:58.018494 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da41074-7e56-445a-a6a4-fb8b9aad2b6f None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:58.021823 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da41074-7e56-445a-a6a4-fb8b9aad2b6f None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:58.044345 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-3da41074-7e56-445a-a6a4-fb8b9aad2b6f None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 29 11:06:58.044345 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:06:58.044345 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:06:58.044345 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:06:58.044345 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:06:58.044345 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:06:58.044345 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:06:58.044345 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:06:58.044345 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:06:58.044345 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:06:58.044345 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:06:58.044345 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:06:58.044345 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 11:06:58.044345 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 11:06:58.044345 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 11:06:58.044345 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 11:06:58.045546 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:06:58.045546 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:06:58.045546 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 11:06:58.045546 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 11:06:58.045546 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 11:06:58.045546 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 11:06:58.045546 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:06:58.045546 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:06:58.045546 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:06:58.045546 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:06:58.045546 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:06:58.045546 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jun 29 11:06:58.045546 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:06:58.045546 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:06:58.045546 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:06:58.045546 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:06:58.045546 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:06:58.045546 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:06:58.046704 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:06:58.046704 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:06:58.046704 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 11:06:58.046704 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 11:06:58.046704 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 11:06:58.046704 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 11:06:58.046704 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 11:06:58.046704 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 11:06:58.046704 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 29 11:06:58.046704 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:06:58.049428 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 327/656] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Jun 29 11:06:57 2022] GET /identity/v3/users/cinder => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:06:58.050996 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da41074-7e56-445a-a6a4-fb8b9aad2b6f None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:58.101998 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb011af-6ae2-41d4-a744-557d5301350c None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:58.137126 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38f97fb-6bb8-4734-a190-8168dc8bc8ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:58.138003 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38f97fb-6bb8-4734-a190-8168dc8bc8ec None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:58.138389 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38f97fb-6bb8-4734-a190-8168dc8bc8ec None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:58.138769 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38f97fb-6bb8-4734-a190-8168dc8bc8ec None admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:58.139605 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38f97fb-6bb8-4734-a190-8168dc8bc8ec None admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:06:58.140033 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38f97fb-6bb8-4734-a190-8168dc8bc8ec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:58.141516 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38f97fb-6bb8-4734-a190-8168dc8bc8ec None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:58.145636 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:58.155679 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 330/657] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Jun 29 11:06:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:58.157300 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38f97fb-6bb8-4734-a190-8168dc8bc8ec None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:58.183670 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da41074-7e56-445a-a6a4-fb8b9aad2b6f None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:58.219544 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa0afdb-0547-4d43-a0ab-7646cfe8c06f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:58.220544 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa0afdb-0547-4d43-a0ab-7646cfe8c06f None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:58.220668 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa0afdb-0547-4d43-a0ab-7646cfe8c06f None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:58.221049 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa0afdb-0547-4d43-a0ab-7646cfe8c06f None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:58.221977 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa0afdb-0547-4d43-a0ab-7646cfe8c06f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:06:58.222334 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa0afdb-0547-4d43-a0ab-7646cfe8c06f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:58.224441 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa0afdb-0547-4d43-a0ab-7646cfe8c06f None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:58.236615 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:58.237801 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 328/658] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Jun 29 11:06:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:58.238757 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa0afdb-0547-4d43-a0ab-7646cfe8c06f None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:58.283235 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38f97fb-6bb8-4734-a190-8168dc8bc8ec None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:58.320404 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0803c583-7bd9-4318-824e-81f1539c73a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:58.321333 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0803c583-7bd9-4318-824e-81f1539c73a3 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:58.322020 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0803c583-7bd9-4318-824e-81f1539c73a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:58.322513 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0803c583-7bd9-4318-824e-81f1539c73a3 None admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:58.323578 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0803c583-7bd9-4318-824e-81f1539c73a3 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:06:58.324153 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0803c583-7bd9-4318-824e-81f1539c73a3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:58.325825 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0803c583-7bd9-4318-824e-81f1539c73a3 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:58.345254 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 331/659] 10.222.0.44 () {56 vars in 1137 bytes} [Wed Jun 29 11:06:58 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:58.346664 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0803c583-7bd9-4318-824e-81f1539c73a3 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:58.346845 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa0afdb-0547-4d43-a0ab-7646cfe8c06f None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:58.375034 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:58.385018 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f265b5c0-0b7c-4fcc-be5c-e95433e6773c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:58.385869 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f265b5c0-0b7c-4fcc-be5c-e95433e6773c None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:58.386154 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f265b5c0-0b7c-4fcc-be5c-e95433e6773c None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:58.386476 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f265b5c0-0b7c-4fcc-be5c-e95433e6773c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:58.393675 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f265b5c0-0b7c-4fcc-be5c-e95433e6773c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:58.395774 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f265b5c0-0b7c-4fcc-be5c-e95433e6773c None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:58.411218 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-f265b5c0-0b7c-4fcc-be5c-e95433e6773c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 11:06:58.411218 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:06:58.411218 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:06:58.411218 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:06:58.411218 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:06:58.411218 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:06:58.411218 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:06:58.411218 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:06:58.411218 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:06:58.411218 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:06:58.411218 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:06:58.411218 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:06:58.411218 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 11:06:58.411218 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 11:06:58.411218 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 11:06:58.412161 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 11:06:58.412161 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:06:58.412161 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:06:58.412161 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:06:58.412161 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:06:58.412161 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:06:58.412161 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:06:58.412161 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:06:58.412161 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jun 29 11:06:58.412161 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:06:58.412161 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:06:58.412161 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:06:58.412161 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:06:58.412161 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:06:58.412161 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:06:58.412161 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:06:58.412161 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:06:58.413644 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 11:06:58.413644 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 11:06:58.413644 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 11:06:58.413644 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 11:06:58.413644 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 11:06:58.413644 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 11:06:58.413644 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 11:06:58.413644 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:06:58.413644 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 329/660] 10.222.0.44 () {56 vars in 1111 bytes} [Wed Jun 29 11:06:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:06:58.413644 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f265b5c0-0b7c-4fcc-be5c-e95433e6773c None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:58.458418 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0803c583-7bd9-4318-824e-81f1539c73a3 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:58.477305 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:58.495171 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62b688d-8119-45c6-aea2-4e503ad163b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:58.496314 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62b688d-8119-45c6-aea2-4e503ad163b5 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:58.496807 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62b688d-8119-45c6-aea2-4e503ad163b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:58.497289 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62b688d-8119-45c6-aea2-4e503ad163b5 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:58.505548 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62b688d-8119-45c6-aea2-4e503ad163b5 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:58.507258 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62b688d-8119-45c6-aea2-4e503ad163b5 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:58.523750 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-b62b688d-8119-45c6-aea2-4e503ad163b5 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 29 11:06:58.523750 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:06:58.523750 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:06:58.523750 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:06:58.523750 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:06:58.523750 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:06:58.523750 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:06:58.523750 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:06:58.523750 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:06:58.523750 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:06:58.523750 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:06:58.523750 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:06:58.523750 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 11:06:58.523750 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 11:06:58.523750 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 11:06:58.523750 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 11:06:58.524739 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:06:58.524739 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:06:58.524739 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 11:06:58.524739 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 11:06:58.524739 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 11:06:58.524739 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 11:06:58.524739 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:06:58.524739 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:06:58.524739 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:06:58.524739 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:06:58.524739 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:06:58.524739 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jun 29 11:06:58.524739 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:06:58.524739 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:06:58.524739 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:06:58.524739 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:06:58.524739 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:06:58.524739 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:06:58.525680 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:06:58.525680 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:06:58.525680 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 11:06:58.525680 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 11:06:58.525680 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 11:06:58.525680 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 11:06:58.525680 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 11:06:58.525680 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 11:06:58.525680 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 29 11:06:58.525680 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jun 29 11:06:58.526319 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f265b5c0-0b7c-4fcc-be5c-e95433e6773c None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:58.528531 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 332/661] 10.222.0.44 () {56 vars in 1102 bytes} [Wed Jun 29 11:06:58 2022] GET /identity/v3/users/neutron => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:06:58.529583 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62b688d-8119-45c6-aea2-4e503ad163b5 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:58.560412 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bec500-01f3-416f-b2a1-384ad905ba5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:58.561059 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bec500-01f3-416f-b2a1-384ad905ba5d None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:58.561383 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bec500-01f3-416f-b2a1-384ad905ba5d None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:58.561667 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bec500-01f3-416f-b2a1-384ad905ba5d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:58.620742 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:58.635350 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bec500-01f3-416f-b2a1-384ad905ba5d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:58.640289 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62b688d-8119-45c6-aea2-4e503ad163b5 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:58.640402 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bec500-01f3-416f-b2a1-384ad905ba5d None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:58.673121 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4152a10-452a-4d58-ba31-765c9846a19f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:58.674143 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4152a10-452a-4d58-ba31-765c9846a19f None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:58.674634 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4152a10-452a-4d58-ba31-765c9846a19f None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:58.675148 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4152a10-452a-4d58-ba31-765c9846a19f None admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:58.676217 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4152a10-452a-4d58-ba31-765c9846a19f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:06:58.676734 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4152a10-452a-4d58-ba31-765c9846a19f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:58.678368 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4152a10-452a-4d58-ba31-765c9846a19f None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:58.691030 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 333/662] 10.222.0.44 () {56 vars in 1148 bytes} [Wed Jun 29 11:06:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:58.692834 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4152a10-452a-4d58-ba31-765c9846a19f None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:58.736117 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:58.748621 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 330/663] 10.222.0.44 () {58 vars in 1282 bytes} [Wed Jun 29 11:06:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 29 11:06:58.750640 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ab28ae-1d13-4e64-a44a-011e401fc6e4 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:58.750640 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ab28ae-1d13-4e64-a44a-011e401fc6e4 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:58.751135 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ab28ae-1d13-4e64-a44a-011e401fc6e4 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:58.752953 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 331/664] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:06:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:06:58.753934 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ab28ae-1d13-4e64-a44a-011e401fc6e4 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:58.806039 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4152a10-452a-4d58-ba31-765c9846a19f None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:58.839966 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bb982f-37e2-43a3-b17e-cce206357f61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:58.840628 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bb982f-37e2-43a3-b17e-cce206357f61 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:58.840915 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bb982f-37e2-43a3-b17e-cce206357f61 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:58.841190 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bb982f-37e2-43a3-b17e-cce206357f61 None admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:58.842011 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bb982f-37e2-43a3-b17e-cce206357f61 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:06:58.842317 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bb982f-37e2-43a3-b17e-cce206357f61 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:58.843725 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bb982f-37e2-43a3-b17e-cce206357f61 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:58.846132 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:58.861545 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 334/665] 10.222.0.44 () {56 vars in 1139 bytes} [Wed Jun 29 11:06:58 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:58.863558 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ab28ae-1d13-4e64-a44a-011e401fc6e4 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:58.867076 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bb982f-37e2-43a3-b17e-cce206357f61 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:58.901765 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38d9e7d-3c75-450c-bbfb-c16c692a8fdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:58.902427 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38d9e7d-3c75-450c-bbfb-c16c692a8fdf None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:58.902709 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38d9e7d-3c75-450c-bbfb-c16c692a8fdf None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:58.902988 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38d9e7d-3c75-450c-bbfb-c16c692a8fdf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:58.903816 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38d9e7d-3c75-450c-bbfb-c16c692a8fdf None admin] RBAC: Adding query filter params (role.id=a8212be3426741cbb1b38b1646c66405, scope.project.id=ebe5dffa4bb8440286687daefb6c98f4, user.id=ede3a17c7063432e8ee66462766584f8) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:06:58.904157 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38d9e7d-3c75-450c-bbfb-c16c692a8fdf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:58.905571 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38d9e7d-3c75-450c-bbfb-c16c692a8fdf None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:58.915966 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 332/666] 10.222.0.44 () {56 vars in 1374 bytes} [Wed Jun 29 11:06:58 2022] GET /identity/v3/role_assignments?user.id=ede3a17c7063432e8ee66462766584f8&scope.project.id=ebe5dffa4bb8440286687daefb6c98f4&role.id=a8212be3426741cbb1b38b1646c66405 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:58.915966 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671c194e-f57f-4454-95db-df1c106408d0 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:58.915966 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671c194e-f57f-4454-95db-df1c106408d0 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:58.915966 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671c194e-f57f-4454-95db-df1c106408d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:58.933943 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-671c194e-f57f-4454-95db-df1c106408d0 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:06:58.935426 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-671c194e-f57f-4454-95db-df1c106408d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:06:58.993694 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:59.001027 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bb982f-37e2-43a3-b17e-cce206357f61 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:59.027859 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 333/667] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:06:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 11:06:59.027859 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671c194e-f57f-4454-95db-df1c106408d0 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:59.064915 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01f9b52-1911-4bc0-b255-55fe4d49335a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:59.066110 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01f9b52-1911-4bc0-b255-55fe4d49335a None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:59.066658 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01f9b52-1911-4bc0-b255-55fe4d49335a None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:59.067729 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01f9b52-1911-4bc0-b255-55fe4d49335a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:59.079388 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01f9b52-1911-4bc0-b255-55fe4d49335a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:59.081207 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01f9b52-1911-4bc0-b255-55fe4d49335a None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:59.101219 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-c01f9b52-1911-4bc0-b255-55fe4d49335a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 11:06:59.101219 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:06:59.101219 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:06:59.101219 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:06:59.101219 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:06:59.101219 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:06:59.101219 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:06:59.101219 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:06:59.101219 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:06:59.101219 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:06:59.101219 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:06:59.101219 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:06:59.101219 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 11:06:59.101219 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 11:06:59.101219 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 11:06:59.102188 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 11:06:59.102188 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:06:59.102188 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:06:59.102188 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:06:59.102188 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:06:59.102188 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:06:59.102188 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:06:59.102188 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:06:59.102188 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jun 29 11:06:59.102188 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:06:59.102188 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:06:59.102188 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:06:59.102188 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:06:59.102188 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:06:59.102188 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:06:59.102188 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:06:59.102188 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:06:59.103072 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 11:06:59.103072 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 11:06:59.103072 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 11:06:59.103072 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jun 29 11:06:59.105168 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 335/668] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jun 29 11:06:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:06:59.106226 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01f9b52-1911-4bc0-b255-55fe4d49335a None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:59.119356 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:59.143855 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671c194e-f57f-4454-95db-df1c106408d0 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:59.198171 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63153432-47f3-4cc2-8c87-6d6d5e968d06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:59.198835 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63153432-47f3-4cc2-8c87-6d6d5e968d06 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:59.199120 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63153432-47f3-4cc2-8c87-6d6d5e968d06 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:59.199394 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63153432-47f3-4cc2-8c87-6d6d5e968d06 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:59.231020 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01f9b52-1911-4bc0-b255-55fe4d49335a None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:59.266804 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99a2228-117b-4b01-9f16-84d41eb6f46c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:59.267519 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99a2228-117b-4b01-9f16-84d41eb6f46c None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:59.267797 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99a2228-117b-4b01-9f16-84d41eb6f46c None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:59.268094 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99a2228-117b-4b01-9f16-84d41eb6f46c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:59.279142 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99a2228-117b-4b01-9f16-84d41eb6f46c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 11:06:59.286978 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-b99a2228-117b-4b01-9f16-84d41eb6f46c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 29 11:06:59.286978 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:06:59.286978 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:06:59.286978 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:06:59.286978 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:06:59.286978 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:06:59.286978 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:06:59.286978 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:06:59.286978 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:06:59.286978 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:06:59.286978 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:06:59.286978 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:06:59.286978 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 11:06:59.286978 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 11:06:59.286978 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 11:06:59.286978 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 11:06:59.287934 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 11:06:59.287934 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 11:06:59.287934 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 11:06:59.287934 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 11:06:59.287934 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:06:59.287934 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:06:59.287934 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:06:59.287934 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:06:59.287934 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:06:59.287934 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:06:59.287934 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:06:59.287934 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jun 29 11:06:59.287934 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:06:59.287934 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:06:59.287934 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:06:59.287934 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:06:59.288774 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:06:59.288774 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:06:59.288774 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:06:59.288774 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:06:59.288774 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 11:06:59.288774 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 11:06:59.288774 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 11:06:59.288774 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 11:06:59.288774 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 11:06:59.288774 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 11:06:59.288774 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 29 11:06:59.288774 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jun 29 11:06:59.288774 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 336/669] 10.222.0.44 () {56 vars in 1102 bytes} [Wed Jun 29 11:06:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:06:59.290003 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99a2228-117b-4b01-9f16-84d41eb6f46c None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:59.295917 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63153432-47f3-4cc2-8c87-6d6d5e968d06 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:59.297414 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63153432-47f3-4cc2-8c87-6d6d5e968d06 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:59.300457 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:59.388202 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:59.394086 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99a2228-117b-4b01-9f16-84d41eb6f46c None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:59.411590 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 334/670] 10.222.0.44 () {58 vars in 1282 bytes} [Wed Jun 29 11:06:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 29 11:06:59.412479 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63153432-47f3-4cc2-8c87-6d6d5e968d06 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:59.434689 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d9667e-35c4-419d-9ea5-57a7af1589ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:59.435446 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d9667e-35c4-419d-9ea5-57a7af1589ae None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:59.435797 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d9667e-35c4-419d-9ea5-57a7af1589ae None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:59.436162 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d9667e-35c4-419d-9ea5-57a7af1589ae None admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:59.437010 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d9667e-35c4-419d-9ea5-57a7af1589ae None admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:06:59.437376 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d9667e-35c4-419d-9ea5-57a7af1589ae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:59.438829 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d9667e-35c4-419d-9ea5-57a7af1589ae None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:59.452007 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 337/671] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Jun 29 11:06:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:59.452969 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d9667e-35c4-419d-9ea5-57a7af1589ae None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:59.521444 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63153432-47f3-4cc2-8c87-6d6d5e968d06 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:59.534296 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:59.572023 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d62cccc-8942-474e-b098-ba0c48328fa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:59.572701 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d62cccc-8942-474e-b098-ba0c48328fa3 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:59.572989 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d62cccc-8942-474e-b098-ba0c48328fa3 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:59.573267 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d62cccc-8942-474e-b098-ba0c48328fa3 None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:59.573471 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d9667e-35c4-419d-9ea5-57a7af1589ae None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:59.574013 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d62cccc-8942-474e-b098-ba0c48328fa3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:06:59.574322 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d62cccc-8942-474e-b098-ba0c48328fa3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:59.576889 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d62cccc-8942-474e-b098-ba0c48328fa3 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:59.587802 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 335/672] 10.222.0.44 () {56 vars in 1103 bytes} [Wed Jun 29 11:06:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:59.588655 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d62cccc-8942-474e-b098-ba0c48328fa3 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:59.614886 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349d1f3d-f493-44f3-aa8d-d8be7341c8a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:59.615685 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349d1f3d-f493-44f3-aa8d-d8be7341c8a8 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:59.616004 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349d1f3d-f493-44f3-aa8d-d8be7341c8a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:59.616282 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349d1f3d-f493-44f3-aa8d-d8be7341c8a8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:59.627042 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349d1f3d-f493-44f3-aa8d-d8be7341c8a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:59.628596 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349d1f3d-f493-44f3-aa8d-d8be7341c8a8 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:59.643488 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:59.650192 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-349d1f3d-f493-44f3-aa8d-d8be7341c8a8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 11:06:59.650192 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:06:59.650192 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:06:59.650192 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:06:59.650192 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:06:59.650192 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:06:59.650192 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:06:59.650192 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:06:59.650192 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:06:59.650192 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:06:59.650192 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:06:59.650192 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:06:59.650192 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 11:06:59.650192 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 11:06:59.650192 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 11:06:59.651281 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 11:06:59.651281 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:06:59.651281 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:06:59.651281 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:06:59.651281 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:06:59.651281 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:06:59.651281 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:06:59.651281 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:06:59.651281 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jun 29 11:06:59.651281 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:06:59.651281 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:06:59.651281 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:06:59.651281 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:06:59.651281 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:06:59.651281 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:06:59.651281 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:06:59.651281 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:06:59.652405 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 11:06:59.652405 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 11:06:59.652405 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 11:06:59.652405 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jun 29 11:06:59.652405 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 338/673] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jun 29 11:06:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:06:59.653528 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349d1f3d-f493-44f3-aa8d-d8be7341c8a8 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:59.707017 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d62cccc-8942-474e-b098-ba0c48328fa3 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:59.742582 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59736b3-602b-4f8a-b87d-8247bda676ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:59.743380 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59736b3-602b-4f8a-b87d-8247bda676ba None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:59.743672 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59736b3-602b-4f8a-b87d-8247bda676ba None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:59.743979 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59736b3-602b-4f8a-b87d-8247bda676ba None admin] PATH_INFO: `/v3/projects/service` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:59.751116 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59736b3-602b-4f8a-b87d-8247bda676ba None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:59.752576 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59736b3-602b-4f8a-b87d-8247bda676ba None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:59.763778 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349d1f3d-f493-44f3-aa8d-d8be7341c8a8 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:59.767450 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-e59736b3-602b-4f8a-b87d-8247bda676ba None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 11:06:59.767450 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:06:59.767450 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:06:59.767450 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:06:59.767450 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:06:59.767450 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:06:59.767450 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:06:59.767450 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:06:59.767450 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:06:59.767450 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:06:59.767450 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:06:59.767450 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:06:59.767450 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 11:06:59.767450 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 11:06:59.767450 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 11:06:59.768513 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 11:06:59.768513 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:06:59.768513 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:06:59.768513 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:06:59.768513 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:06:59.768513 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:06:59.768513 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:06:59.768513 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:06:59.768513 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jun 29 11:06:59.768513 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:06:59.768513 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:06:59.768513 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:06:59.768513 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:06:59.768513 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:06:59.768513 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:06:59.768513 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:06:59.768513 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:06:59.769413 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 11:06:59.769413 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 11:06:59.769413 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 11:06:59.769413 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 11:06:59.769413 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 11:06:59.769413 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 11:06:59.769413 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 11:06:59.769413 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:06:59.773851 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 336/674] 10.222.0.44 () {56 vars in 1111 bytes} [Wed Jun 29 11:06:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:06:59.774640 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb397d7-2b3c-4d3a-9dc7-614b6bb61fb3 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:59.774937 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb397d7-2b3c-4d3a-9dc7-614b6bb61fb3 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:59.775209 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb397d7-2b3c-4d3a-9dc7-614b6bb61fb3 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:59.776540 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 337/675] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:06:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:06:59.777230 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb397d7-2b3c-4d3a-9dc7-614b6bb61fb3 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:59.789207 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:59.804301 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07fb4ef-556f-48ea-84b3-8d304d8d98e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:59.805094 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07fb4ef-556f-48ea-84b3-8d304d8d98e3 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:59.805384 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07fb4ef-556f-48ea-84b3-8d304d8d98e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:59.805692 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07fb4ef-556f-48ea-84b3-8d304d8d98e3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:59.815703 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07fb4ef-556f-48ea-84b3-8d304d8d98e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:59.817287 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07fb4ef-556f-48ea-84b3-8d304d8d98e3 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:59.836509 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-e07fb4ef-556f-48ea-84b3-8d304d8d98e3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 11:06:59.836509 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:06:59.836509 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:06:59.836509 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:06:59.836509 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:06:59.836509 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:06:59.836509 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:06:59.836509 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:06:59.836509 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:06:59.836509 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:06:59.836509 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:06:59.836509 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:06:59.836509 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 11:06:59.836509 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 11:06:59.836509 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 11:06:59.837438 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 11:06:59.837438 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:06:59.837438 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:06:59.837438 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:06:59.837438 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:06:59.837438 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:06:59.837438 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:06:59.837438 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:06:59.837438 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jun 29 11:06:59.837438 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:06:59.837438 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:06:59.837438 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:06:59.837438 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:06:59.837438 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:06:59.837438 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:06:59.837438 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:06:59.837438 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:06:59.838326 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 11:06:59.838326 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 11:06:59.838326 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 11:06:59.838326 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jun 29 11:06:59.838326 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 339/676] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jun 29 11:06:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:06:59.838608 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07fb4ef-556f-48ea-84b3-8d304d8d98e3 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:06:59.892401 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb397d7-2b3c-4d3a-9dc7-614b6bb61fb3 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:59.894087 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:06:59.933956 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467982a5-35d0-4382-9315-ecce961316a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:59.934625 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467982a5-35d0-4382-9315-ecce961316a1 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:59.934937 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467982a5-35d0-4382-9315-ecce961316a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:59.935225 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467982a5-35d0-4382-9315-ecce961316a1 None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:59.935975 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467982a5-35d0-4382-9315-ecce961316a1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:06:59.936297 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467982a5-35d0-4382-9315-ecce961316a1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:06:59.937648 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467982a5-35d0-4382-9315-ecce961316a1 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:06:59.945522 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07fb4ef-556f-48ea-84b3-8d304d8d98e3 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:06:59.952159 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 338/677] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Jun 29 11:06:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:06:59.953523 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299ef9a3-5bed-4008-be3d-7836a9c98127 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:59.953817 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299ef9a3-5bed-4008-be3d-7836a9c98127 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:59.954100 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299ef9a3-5bed-4008-be3d-7836a9c98127 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:59.971756 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-299ef9a3-5bed-4008-be3d-7836a9c98127 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:06:59.973523 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-299ef9a3-5bed-4008-be3d-7836a9c98127 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:06:59.996667 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23f9fc3-17ca-46d7-89ec-f8d801a4f257 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:06:59.997318 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23f9fc3-17ca-46d7-89ec-f8d801a4f257 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:06:59.998021 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23f9fc3-17ca-46d7-89ec-f8d801a4f257 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:06:59.998303 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23f9fc3-17ca-46d7-89ec-f8d801a4f257 None admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:06:59.999084 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23f9fc3-17ca-46d7-89ec-f8d801a4f257 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:06:59.999420 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23f9fc3-17ca-46d7-89ec-f8d801a4f257 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:00.000906 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23f9fc3-17ca-46d7-89ec-f8d801a4f257 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:00.018601 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 340/678] 10.222.0.44 () {56 vars in 1148 bytes} [Wed Jun 29 11:06:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:07:00.018601 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23f9fc3-17ca-46d7-89ec-f8d801a4f257 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:00.045763 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 339/679] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:06:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 11:07:00.046533 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299ef9a3-5bed-4008-be3d-7836a9c98127 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:00.061495 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:00.135123 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23f9fc3-17ca-46d7-89ec-f8d801a4f257 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:00.156593 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:00.167871 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299ef9a3-5bed-4008-be3d-7836a9c98127 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:00.172847 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e099cfa-143a-448b-bb58-0ce95aade3bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:00.173508 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e099cfa-143a-448b-bb58-0ce95aade3bc None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:00.173791 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e099cfa-143a-448b-bb58-0ce95aade3bc None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:00.174065 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e099cfa-143a-448b-bb58-0ce95aade3bc None admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:00.174775 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e099cfa-143a-448b-bb58-0ce95aade3bc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:07:00.175077 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e099cfa-143a-448b-bb58-0ce95aade3bc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:00.176600 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e099cfa-143a-448b-bb58-0ce95aade3bc None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:00.189018 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 341/680] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Jun 29 11:07:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:07:00.190123 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e099cfa-143a-448b-bb58-0ce95aade3bc None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:00.210709 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5d89c6-8262-4de5-9299-e241b8fc3964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:00.211561 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5d89c6-8262-4de5-9299-e241b8fc3964 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:00.211780 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5d89c6-8262-4de5-9299-e241b8fc3964 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:00.212086 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5d89c6-8262-4de5-9299-e241b8fc3964 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:00.218905 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5d89c6-8262-4de5-9299-e241b8fc3964 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:00.220326 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5d89c6-8262-4de5-9299-e241b8fc3964 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:00.234660 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-fa5d89c6-8262-4de5-9299-e241b8fc3964 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 11:07:00.234660 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:07:00.234660 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:07:00.234660 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:07:00.234660 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:07:00.234660 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:07:00.234660 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:07:00.234660 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:07:00.234660 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:07:00.234660 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:07:00.234660 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:07:00.234660 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:07:00.234660 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 11:07:00.234660 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 11:07:00.234660 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 11:07:00.235558 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 11:07:00.235558 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:07:00.235558 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:07:00.235558 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:07:00.235558 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:07:00.235558 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:07:00.235558 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:07:00.235558 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:07:00.235558 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jun 29 11:07:00.235558 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:07:00.235558 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:07:00.235558 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:07:00.235558 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:07:00.235558 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:07:00.235558 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:07:00.235558 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:07:00.235558 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:07:00.237548 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 11:07:00.237548 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 11:07:00.237548 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 11:07:00.237548 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 11:07:00.237548 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 11:07:00.237548 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 11:07:00.237548 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 11:07:00.237548 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:07:00.237548 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 340/681] 10.222.0.44 () {56 vars in 1111 bytes} [Wed Jun 29 11:07:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:07:00.237548 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5d89c6-8262-4de5-9299-e241b8fc3964 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:00.290908 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:00.303289 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e099cfa-143a-448b-bb58-0ce95aade3bc None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:00.336552 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f122ee6c-821a-4f66-80c4-7a278c7afa9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:00.337434 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f122ee6c-821a-4f66-80c4-7a278c7afa9f None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:00.337865 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f122ee6c-821a-4f66-80c4-7a278c7afa9f None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:00.338287 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f122ee6c-821a-4f66-80c4-7a278c7afa9f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:00.339373 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f122ee6c-821a-4f66-80c4-7a278c7afa9f None admin] RBAC: Adding query filter params (role.id=a8212be3426741cbb1b38b1646c66405, scope.project.id=ebe5dffa4bb8440286687daefb6c98f4, user.id=ff7931b75a924394b1af733febce0834) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:07:00.339991 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f122ee6c-821a-4f66-80c4-7a278c7afa9f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:00.341420 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f122ee6c-821a-4f66-80c4-7a278c7afa9f None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:00.355856 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 342/682] 10.222.0.44 () {56 vars in 1374 bytes} [Wed Jun 29 11:07:00 2022] GET /identity/v3/role_assignments?user.id=ff7931b75a924394b1af733febce0834&scope.project.id=ebe5dffa4bb8440286687daefb6c98f4&role.id=a8212be3426741cbb1b38b1646c66405 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:07:00.356067 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5d89c6-8262-4de5-9299-e241b8fc3964 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:00.356187 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f122ee6c-821a-4f66-80c4-7a278c7afa9f None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:00.394581 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cd6b3e-fd69-4f7d-b966-f4ea71d6acb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:00.395520 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cd6b3e-fd69-4f7d-b966-f4ea71d6acb6 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:00.395960 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cd6b3e-fd69-4f7d-b966-f4ea71d6acb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:00.396244 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cd6b3e-fd69-4f7d-b966-f4ea71d6acb6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:00.408465 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cd6b3e-fd69-4f7d-b966-f4ea71d6acb6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 11:07:00.416522 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-c5cd6b3e-fd69-4f7d-b966-f4ea71d6acb6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 11:07:00.416522 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:07:00.416522 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:07:00.416522 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:07:00.416522 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:07:00.416522 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:07:00.416522 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:07:00.416522 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:07:00.416522 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:07:00.416522 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:07:00.416522 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:07:00.416522 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:07:00.416522 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 11:07:00.416522 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 11:07:00.416522 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 11:07:00.416522 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 11:07:00.417672 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 11:07:00.417672 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 11:07:00.417672 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 11:07:00.417672 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 11:07:00.417672 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:07:00.417672 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:07:00.417672 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:07:00.417672 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:07:00.417672 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:07:00.417672 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:07:00.417672 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:07:00.417672 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jun 29 11:07:00.417672 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:07:00.417672 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:07:00.417672 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:07:00.417672 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:07:00.418526 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:07:00.418526 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:07:00.418526 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:07:00.418526 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:07:00.418526 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 11:07:00.418526 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 11:07:00.418526 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 11:07:00.418526 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 11:07:00.418526 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 11:07:00.418526 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 11:07:00.418526 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 11:07:00.418526 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:07:00.418526 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 341/683] 10.222.0.44 () {56 vars in 1096 bytes} [Wed Jun 29 11:07:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:07:00.419263 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cd6b3e-fd69-4f7d-b966-f4ea71d6acb6 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:00.436730 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:00.491943 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f122ee6c-821a-4f66-80c4-7a278c7afa9f None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:00.533554 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2f7d1f-791a-4b08-87fa-5306c7247f69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:00.536097 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2f7d1f-791a-4b08-87fa-5306c7247f69 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:00.536097 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2f7d1f-791a-4b08-87fa-5306c7247f69 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:00.536097 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2f7d1f-791a-4b08-87fa-5306c7247f69 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:00.538491 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cd6b3e-fd69-4f7d-b966-f4ea71d6acb6 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:00.545251 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2f7d1f-791a-4b08-87fa-5306c7247f69 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:00.546709 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2f7d1f-791a-4b08-87fa-5306c7247f69 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:00.565486 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-5f2f7d1f-791a-4b08-87fa-5306c7247f69 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 29 11:07:00.565486 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:07:00.565486 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:07:00.565486 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:07:00.565486 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:07:00.565486 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:07:00.565486 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:07:00.565486 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:07:00.565486 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:07:00.565486 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:07:00.565486 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:07:00.565486 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:07:00.565486 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 11:07:00.565486 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 11:07:00.565486 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 11:07:00.565486 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 11:07:00.566488 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:07:00.566488 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:07:00.566488 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 11:07:00.566488 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 11:07:00.566488 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 11:07:00.566488 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 11:07:00.566488 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:07:00.566488 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:07:00.566488 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:07:00.566488 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:07:00.566488 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:07:00.566488 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jun 29 11:07:00.566488 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:07:00.566488 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:07:00.566488 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:07:00.566488 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:07:00.566488 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:07:00.566488 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:07:00.568488 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:07:00.568488 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:07:00.568488 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 11:07:00.568488 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 11:07:00.568488 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 11:07:00.568488 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 11:07:00.568488 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 11:07:00.568488 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 11:07:00.568488 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 29 11:07:00.568488 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jun 29 11:07:00.568488 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 343/684] 10.222.0.44 () {56 vars in 1096 bytes} [Wed Jun 29 11:07:00 2022] GET /identity/v3/users/swift => generated 83 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:07:00.568488 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2f7d1f-791a-4b08-87fa-5306c7247f69 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:00.583191 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7411aa8f-1001-4c35-898d-cf21cb265964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:00.584337 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7411aa8f-1001-4c35-898d-cf21cb265964 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:00.584863 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7411aa8f-1001-4c35-898d-cf21cb265964 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:00.585392 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7411aa8f-1001-4c35-898d-cf21cb265964 None admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:00.586430 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7411aa8f-1001-4c35-898d-cf21cb265964 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:07:00.586959 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7411aa8f-1001-4c35-898d-cf21cb265964 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:00.587689 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:00.589946 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7411aa8f-1001-4c35-898d-cf21cb265964 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:00.606392 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 342/685] 10.222.0.44 () {56 vars in 1148 bytes} [Wed Jun 29 11:07:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:07:00.607723 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7411aa8f-1001-4c35-898d-cf21cb265964 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:00.691522 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2f7d1f-791a-4b08-87fa-5306c7247f69 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:00.697303 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:00.731325 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1623878-4776-4891-a7ab-c3c5fd236b9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:00.732073 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1623878-4776-4891-a7ab-c3c5fd236b9d None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:00.732567 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1623878-4776-4891-a7ab-c3c5fd236b9d None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:00.732991 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1623878-4776-4891-a7ab-c3c5fd236b9d None admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:00.734086 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1623878-4776-4891-a7ab-c3c5fd236b9d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:07:00.734399 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1623878-4776-4891-a7ab-c3c5fd236b9d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:00.736248 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1623878-4776-4891-a7ab-c3c5fd236b9d None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:00.740504 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7411aa8f-1001-4c35-898d-cf21cb265964 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:00.752273 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 344/686] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Jun 29 11:07:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:07:00.754140 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1623878-4776-4891-a7ab-c3c5fd236b9d None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:00.793227 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43db962c-6116-4c08-ad19-10d08f908047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:00.793903 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43db962c-6116-4c08-ad19-10d08f908047 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:00.794184 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43db962c-6116-4c08-ad19-10d08f908047 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:00.794514 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43db962c-6116-4c08-ad19-10d08f908047 None admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:00.795608 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43db962c-6116-4c08-ad19-10d08f908047 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:07:00.795946 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43db962c-6116-4c08-ad19-10d08f908047 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:00.798254 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43db962c-6116-4c08-ad19-10d08f908047 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:00.826576 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 343/687] 10.222.0.44 () {56 vars in 1135 bytes} [Wed Jun 29 11:07:00 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:07:00.827403 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43db962c-6116-4c08-ad19-10d08f908047 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:00.850488 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:00.873217 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1623878-4776-4891-a7ab-c3c5fd236b9d None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:00.917680 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2478287d-15a3-424b-9313-79f3f5eae0ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:00.918337 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2478287d-15a3-424b-9313-79f3f5eae0ec None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:00.918618 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2478287d-15a3-424b-9313-79f3f5eae0ec None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:00.918889 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2478287d-15a3-424b-9313-79f3f5eae0ec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:00.919719 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2478287d-15a3-424b-9313-79f3f5eae0ec None admin] RBAC: Adding query filter params (role.id=a8212be3426741cbb1b38b1646c66405, scope.project.id=ebe5dffa4bb8440286687daefb6c98f4, user.id=b5f4bb9f989e4024a5da28dced646b92) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:07:00.920060 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2478287d-15a3-424b-9313-79f3f5eae0ec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:00.922172 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2478287d-15a3-424b-9313-79f3f5eae0ec None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:00.935880 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 345/688] 10.222.0.44 () {56 vars in 1374 bytes} [Wed Jun 29 11:07:00 2022] GET /identity/v3/role_assignments?user.id=b5f4bb9f989e4024a5da28dced646b92&scope.project.id=ebe5dffa4bb8440286687daefb6c98f4&role.id=a8212be3426741cbb1b38b1646c66405 => generated 608 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:07:00.935880 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2478287d-15a3-424b-9313-79f3f5eae0ec None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:00.980241 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43db962c-6116-4c08-ad19-10d08f908047 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:01.014336 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:01.021263 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84b648c-26bb-4cb8-86d7-2c16563921df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:01.022239 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84b648c-26bb-4cb8-86d7-2c16563921df None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:01.022646 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84b648c-26bb-4cb8-86d7-2c16563921df None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:01.022994 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84b648c-26bb-4cb8-86d7-2c16563921df None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:01.034594 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84b648c-26bb-4cb8-86d7-2c16563921df None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:01.036852 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84b648c-26bb-4cb8-86d7-2c16563921df None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:01.056842 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-a84b648c-26bb-4cb8-86d7-2c16563921df None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 11:07:01.056842 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:07:01.056842 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:07:01.056842 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:07:01.056842 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:07:01.056842 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:07:01.056842 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:07:01.056842 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:07:01.056842 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:07:01.056842 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:07:01.056842 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:07:01.056842 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:07:01.056842 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 11:07:01.056842 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 11:07:01.056842 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 11:07:01.058481 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 11:07:01.058481 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:07:01.058481 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:07:01.058481 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:07:01.058481 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:07:01.058481 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:07:01.058481 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:07:01.058481 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:07:01.058481 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jun 29 11:07:01.058481 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:07:01.058481 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:07:01.058481 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:07:01.058481 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:07:01.058481 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:07:01.058481 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:07:01.058481 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:07:01.058481 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:07:01.059675 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 11:07:01.059675 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 11:07:01.059675 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 11:07:01.059675 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:07:01.060729 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 344/689] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jun 29 11:07:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:07:01.066215 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84b648c-26bb-4cb8-86d7-2c16563921df None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:01.141735 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2478287d-15a3-424b-9313-79f3f5eae0ec None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:01.184631 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84b648c-26bb-4cb8-86d7-2c16563921df None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:01.185917 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a10b0e-875e-49fe-aae1-9738ef0192fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:01.186750 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a10b0e-875e-49fe-aae1-9738ef0192fc None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:01.187037 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a10b0e-875e-49fe-aae1-9738ef0192fc None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:01.187317 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a10b0e-875e-49fe-aae1-9738ef0192fc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:01.204012 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:01.229985 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5890d720-71ee-425c-ab6c-50f129170991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:01.231523 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5890d720-71ee-425c-ab6c-50f129170991 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:01.231964 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5890d720-71ee-425c-ab6c-50f129170991 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:01.232262 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5890d720-71ee-425c-ab6c-50f129170991 None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:01.233001 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5890d720-71ee-425c-ab6c-50f129170991 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:07:01.233316 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5890d720-71ee-425c-ab6c-50f129170991 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:01.234698 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5890d720-71ee-425c-ab6c-50f129170991 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:01.250798 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 345/690] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Jun 29 11:07:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 11:07:01.252666 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7e59ab-f3b9-4d7e-88b2-e568cf5cae71 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:01.252969 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7e59ab-f3b9-4d7e-88b2-e568cf5cae71 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:01.253251 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7e59ab-f3b9-4d7e-88b2-e568cf5cae71 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:01.255171 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 346/691] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:07:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:07:01.259669 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7e59ab-f3b9-4d7e-88b2-e568cf5cae71 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:01.268356 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a10b0e-875e-49fe-aae1-9738ef0192fc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:01.270696 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a10b0e-875e-49fe-aae1-9738ef0192fc None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:01.307128 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:01.370626 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7e59ab-f3b9-4d7e-88b2-e568cf5cae71 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:01.372730 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 346/692] 10.222.0.44 () {58 vars in 1282 bytes} [Wed Jun 29 11:07:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 29 11:07:01.374245 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82994c74-8066-4d2b-a24a-8a229f617f76 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:01.374707 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82994c74-8066-4d2b-a24a-8a229f617f76 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:01.375176 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82994c74-8066-4d2b-a24a-8a229f617f76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:01.392100 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-82994c74-8066-4d2b-a24a-8a229f617f76 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:07:01.393593 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-82994c74-8066-4d2b-a24a-8a229f617f76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:07:01.408481 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51e6678-9804-4e9f-89f8-819cb7bb6818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:01.409415 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51e6678-9804-4e9f-89f8-819cb7bb6818 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:01.409749 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51e6678-9804-4e9f-89f8-819cb7bb6818 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:01.410035 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51e6678-9804-4e9f-89f8-819cb7bb6818 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:01.418396 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51e6678-9804-4e9f-89f8-819cb7bb6818 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:01.419905 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51e6678-9804-4e9f-89f8-819cb7bb6818 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:01.434627 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-d51e6678-9804-4e9f-89f8-819cb7bb6818 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 29 11:07:01.434627 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:07:01.434627 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:07:01.434627 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:07:01.434627 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:07:01.434627 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:07:01.434627 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:07:01.434627 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:07:01.434627 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:07:01.434627 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:07:01.434627 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:07:01.434627 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:07:01.434627 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 11:07:01.434627 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 11:07:01.434627 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 11:07:01.434627 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 11:07:01.435800 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:07:01.435800 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:07:01.435800 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 11:07:01.435800 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 11:07:01.435800 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 11:07:01.435800 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 11:07:01.435800 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:07:01.435800 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:07:01.435800 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:07:01.435800 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:07:01.435800 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:07:01.435800 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jun 29 11:07:01.435800 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:07:01.435800 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:07:01.435800 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:07:01.435800 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:07:01.435800 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:07:01.435800 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:07:01.437047 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:07:01.437047 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:07:01.437047 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 11:07:01.437047 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 11:07:01.437047 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 11:07:01.437047 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 11:07:01.437047 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 11:07:01.437047 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 11:07:01.437047 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 29 11:07:01.437047 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:07:01.437047 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 347/693] 10.222.0.44 () {56 vars in 1093 bytes} [Wed Jun 29 11:07:01 2022] GET /identity/v3/users/nova => generated 82 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:07:01.437047 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51e6678-9804-4e9f-89f8-819cb7bb6818 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:01.448179 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:01.469974 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 347/694] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:07:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 11:07:01.474442 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82994c74-8066-4d2b-a24a-8a229f617f76 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:01.577012 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51e6678-9804-4e9f-89f8-819cb7bb6818 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:01.579029 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:01.594404 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82994c74-8066-4d2b-a24a-8a229f617f76 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:01.628018 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12446dfd-06a1-480a-b0d1-7e88870d064c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:01.628819 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12446dfd-06a1-480a-b0d1-7e88870d064c None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:01.629106 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12446dfd-06a1-480a-b0d1-7e88870d064c None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:01.630324 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12446dfd-06a1-480a-b0d1-7e88870d064c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:01.643368 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113543e8-a790-4c7f-89e2-52871d1d9679 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:01.643907 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12446dfd-06a1-480a-b0d1-7e88870d064c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:01.644735 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113543e8-a790-4c7f-89e2-52871d1d9679 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:01.645796 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113543e8-a790-4c7f-89e2-52871d1d9679 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:01.645980 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12446dfd-06a1-480a-b0d1-7e88870d064c None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:01.646510 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113543e8-a790-4c7f-89e2-52871d1d9679 None admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:01.647808 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113543e8-a790-4c7f-89e2-52871d1d9679 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:07:01.648680 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113543e8-a790-4c7f-89e2-52871d1d9679 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:01.650909 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113543e8-a790-4c7f-89e2-52871d1d9679 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:01.668268 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-12446dfd-06a1-480a-b0d1-7e88870d064c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 11:07:01.668268 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:07:01.668268 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:07:01.668268 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:07:01.668268 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:07:01.668268 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:07:01.668268 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:07:01.668268 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:07:01.668268 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:07:01.668268 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:07:01.668268 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:07:01.668268 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:07:01.668268 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 11:07:01.668268 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 11:07:01.668268 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 11:07:01.669262 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 11:07:01.669262 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:07:01.669262 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:07:01.669262 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:07:01.669262 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:07:01.669262 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:07:01.669262 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:07:01.669262 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:07:01.669262 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jun 29 11:07:01.669262 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:07:01.669262 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:07:01.669262 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:07:01.669262 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:07:01.669262 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:07:01.669262 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:07:01.669262 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:07:01.669262 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:07:01.670153 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 11:07:01.670153 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 11:07:01.670153 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 11:07:01.670153 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:07:01.670153 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 348/695] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jun 29 11:07:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:07:01.670886 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12446dfd-06a1-480a-b0d1-7e88870d064c None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:01.680565 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 348/696] 10.222.0.44 () {56 vars in 1133 bytes} [Wed Jun 29 11:07:01 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:07:01.681400 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113543e8-a790-4c7f-89e2-52871d1d9679 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:01.726165 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:01.796862 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12446dfd-06a1-480a-b0d1-7e88870d064c None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:01.810395 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113543e8-a790-4c7f-89e2-52871d1d9679 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:01.846170 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae86b2d-b11e-4efc-bbab-e7f90f3b2379 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:01.846908 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bac27d-6b81-483f-828b-9f6e185f6f69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:01.847060 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae86b2d-b11e-4efc-bbab-e7f90f3b2379 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:01.847236 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae86b2d-b11e-4efc-bbab-e7f90f3b2379 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:01.847517 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae86b2d-b11e-4efc-bbab-e7f90f3b2379 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:01.848754 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bac27d-6b81-483f-828b-9f6e185f6f69 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:01.849195 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bac27d-6b81-483f-828b-9f6e185f6f69 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:01.849621 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bac27d-6b81-483f-828b-9f6e185f6f69 None admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:01.850492 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bac27d-6b81-483f-828b-9f6e185f6f69 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:07:01.850953 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bac27d-6b81-483f-828b-9f6e185f6f69 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:01.852499 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bac27d-6b81-483f-828b-9f6e185f6f69 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:01.859904 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae86b2d-b11e-4efc-bbab-e7f90f3b2379 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:01.861902 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae86b2d-b11e-4efc-bbab-e7f90f3b2379 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:01.868146 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 349/697] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Jun 29 11:07:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:07:01.869146 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bac27d-6b81-483f-828b-9f6e185f6f69 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:01.883187 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-dae86b2d-b11e-4efc-bbab-e7f90f3b2379 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 11:07:01.883187 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:07:01.883187 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:07:01.883187 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:07:01.883187 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:07:01.883187 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:07:01.883187 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:07:01.883187 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:07:01.883187 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:07:01.883187 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:07:01.883187 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:07:01.883187 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:07:01.883187 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 11:07:01.883187 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 11:07:01.883187 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 11:07:01.884336 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 11:07:01.884336 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:07:01.884336 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:07:01.884336 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:07:01.884336 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:07:01.884336 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:07:01.884336 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:07:01.884336 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:07:01.884336 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jun 29 11:07:01.884336 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:07:01.884336 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:07:01.884336 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:07:01.884336 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:07:01.884336 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:07:01.884336 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:07:01.884336 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:07:01.884336 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:07:01.885266 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 11:07:01.885266 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 11:07:01.885266 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 11:07:01.885266 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:07:01.885266 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 349/698] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jun 29 11:07:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:07:01.885549 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae86b2d-b11e-4efc-bbab-e7f90f3b2379 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:01.896937 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:01.987302 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:01.998127 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bac27d-6b81-483f-828b-9f6e185f6f69 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:02.015401 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae86b2d-b11e-4efc-bbab-e7f90f3b2379 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:02.035246 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360b757f-4865-4477-be24-7a5aa145279a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:02.036632 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360b757f-4865-4477-be24-7a5aa145279a None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:02.037018 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360b757f-4865-4477-be24-7a5aa145279a None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:02.037398 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360b757f-4865-4477-be24-7a5aa145279a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:02.061928 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a43b0d7-b45d-442f-9a18-c67033044263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:02.062980 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a43b0d7-b45d-442f-9a18-c67033044263 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:02.063704 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a43b0d7-b45d-442f-9a18-c67033044263 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:02.064023 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a43b0d7-b45d-442f-9a18-c67033044263 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:02.075183 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a43b0d7-b45d-442f-9a18-c67033044263 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:02.076685 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a43b0d7-b45d-442f-9a18-c67033044263 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:02.096890 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-0a43b0d7-b45d-442f-9a18-c67033044263 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 11:07:02.096890 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:07:02.096890 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:07:02.096890 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:07:02.096890 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:07:02.096890 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:07:02.096890 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:07:02.096890 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:07:02.096890 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:07:02.096890 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:07:02.096890 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:07:02.096890 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:07:02.096890 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 11:07:02.096890 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 11:07:02.096890 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 11:07:02.097847 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 11:07:02.097847 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:07:02.097847 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:07:02.097847 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:07:02.097847 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:07:02.097847 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:07:02.097847 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:07:02.097847 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:07:02.097847 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jun 29 11:07:02.097847 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:07:02.097847 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:07:02.097847 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:07:02.097847 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:07:02.097847 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:07:02.097847 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:07:02.097847 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:07:02.097847 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:07:02.098810 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 11:07:02.098810 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 11:07:02.098810 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 11:07:02.098810 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 11:07:02.098810 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 11:07:02.098810 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 11:07:02.098810 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 11:07:02.098810 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:07:02.099286 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 350/699] 10.222.0.44 () {56 vars in 1111 bytes} [Wed Jun 29 11:07:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:07:02.099663 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a43b0d7-b45d-442f-9a18-c67033044263 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:02.114825 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360b757f-4865-4477-be24-7a5aa145279a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:02.116462 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360b757f-4865-4477-be24-7a5aa145279a None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:02.189458 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:02.226573 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:02.236037 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a43b0d7-b45d-442f-9a18-c67033044263 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:02.243341 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 350/700] 10.222.0.44 () {58 vars in 1282 bytes} [Wed Jun 29 11:07:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 29 11:07:02.244387 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360b757f-4865-4477-be24-7a5aa145279a None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:02.279337 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74c5fae-7824-41bb-bfc3-3ba34c4aa647 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:02.281017 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74c5fae-7824-41bb-bfc3-3ba34c4aa647 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:02.281492 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74c5fae-7824-41bb-bfc3-3ba34c4aa647 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:02.282195 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74c5fae-7824-41bb-bfc3-3ba34c4aa647 None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:02.283354 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74c5fae-7824-41bb-bfc3-3ba34c4aa647 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:07:02.284293 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74c5fae-7824-41bb-bfc3-3ba34c4aa647 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:02.286694 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74c5fae-7824-41bb-bfc3-3ba34c4aa647 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:02.307822 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 351/701] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Jun 29 11:07:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:07:02.307822 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74c5fae-7824-41bb-bfc3-3ba34c4aa647 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:02.371603 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360b757f-4865-4477-be24-7a5aa145279a None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:02.406697 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fd3dea-6de3-42c8-9c4e-26feef440743 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:02.407360 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fd3dea-6de3-42c8-9c4e-26feef440743 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:02.407646 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fd3dea-6de3-42c8-9c4e-26feef440743 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:02.407953 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fd3dea-6de3-42c8-9c4e-26feef440743 None admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:02.408881 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fd3dea-6de3-42c8-9c4e-26feef440743 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:07:02.409076 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fd3dea-6de3-42c8-9c4e-26feef440743 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:02.411016 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fd3dea-6de3-42c8-9c4e-26feef440743 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:02.423186 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 351/702] 10.222.0.44 () {56 vars in 1148 bytes} [Wed Jun 29 11:07:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:07:02.424266 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fd3dea-6de3-42c8-9c4e-26feef440743 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:02.438333 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74c5fae-7824-41bb-bfc3-3ba34c4aa647 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:02.467798 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:02.475926 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:02.478304 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f1f46a-89ed-4638-8201-82a2aa8ab58d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:02.479367 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f1f46a-89ed-4638-8201-82a2aa8ab58d None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:02.479864 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f1f46a-89ed-4638-8201-82a2aa8ab58d None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:02.480383 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f1f46a-89ed-4638-8201-82a2aa8ab58d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:02.492160 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f1f46a-89ed-4638-8201-82a2aa8ab58d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:02.493992 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f1f46a-89ed-4638-8201-82a2aa8ab58d None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:02.513640 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-54f1f46a-89ed-4638-8201-82a2aa8ab58d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 11:07:02.513640 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:07:02.513640 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:07:02.513640 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:07:02.513640 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:07:02.513640 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:07:02.513640 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:07:02.513640 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:07:02.513640 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:07:02.513640 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:07:02.513640 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:07:02.513640 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:07:02.513640 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 11:07:02.513640 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 11:07:02.513640 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 11:07:02.515112 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 11:07:02.515112 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:07:02.515112 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:07:02.515112 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:07:02.515112 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:07:02.515112 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:07:02.515112 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:07:02.515112 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:07:02.515112 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jun 29 11:07:02.515112 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:07:02.515112 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:07:02.515112 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:07:02.515112 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:07:02.515112 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:07:02.515112 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:07:02.515112 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:07:02.515112 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:07:02.516099 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 11:07:02.516099 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 11:07:02.516099 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 11:07:02.516099 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:07:02.518236 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 352/703] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jun 29 11:07:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:07:02.519269 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f1f46a-89ed-4638-8201-82a2aa8ab58d None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:02.534108 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fd3dea-6de3-42c8-9c4e-26feef440743 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:02.572433 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc51a8d8-452d-4f1f-925e-87c6f60297b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:02.573316 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc51a8d8-452d-4f1f-925e-87c6f60297b5 None admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:02.573668 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc51a8d8-452d-4f1f-925e-87c6f60297b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:02.574017 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc51a8d8-452d-4f1f-925e-87c6f60297b5 None admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:02.574894 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc51a8d8-452d-4f1f-925e-87c6f60297b5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:02.576450 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc51a8d8-452d-4f1f-925e-87c6f60297b5 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:02.629902 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f1f46a-89ed-4638-8201-82a2aa8ab58d None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:02.671860 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 352/704] 10.222.0.44 () {60 vars in 1132 bytes} [Wed Jun 29 11:07:02 2022] POST /identity/v3/users => generated 254 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:07:02.671860 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc51a8d8-452d-4f1f-925e-87c6f60297b5 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:02.686837 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ea7cd1-e4a1-432b-811b-46b00ed4a230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:02.687551 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ea7cd1-e4a1-432b-811b-46b00ed4a230 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:02.687865 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ea7cd1-e4a1-432b-811b-46b00ed4a230 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:02.688189 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ea7cd1-e4a1-432b-811b-46b00ed4a230 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:02.689051 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ea7cd1-e4a1-432b-811b-46b00ed4a230 None admin] RBAC: Adding query filter params (role.id=a8212be3426741cbb1b38b1646c66405, scope.project.id=ebe5dffa4bb8440286687daefb6c98f4, user.id=e3b38493ba904759badc800e396befda) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:07:02.689367 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ea7cd1-e4a1-432b-811b-46b00ed4a230 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:02.690856 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ea7cd1-e4a1-432b-811b-46b00ed4a230 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:02.708250 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 353/705] 10.222.0.44 () {56 vars in 1374 bytes} [Wed Jun 29 11:07:02 2022] GET /identity/v3/role_assignments?user.id=e3b38493ba904759badc800e396befda&scope.project.id=ebe5dffa4bb8440286687daefb6c98f4&role.id=a8212be3426741cbb1b38b1646c66405 => generated 608 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:07:02.709649 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90a2844-e03f-4f14-8415-5cf444346294 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:02.709940 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90a2844-e03f-4f14-8415-5cf444346294 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:02.710218 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90a2844-e03f-4f14-8415-5cf444346294 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:02.715790 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 354/706] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:07:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:07:02.719214 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f4c69c-9d31-49a1-bd74-62791d74db2c None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:02.719479 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f4c69c-9d31-49a1-bd74-62791d74db2c None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:02.719715 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f4c69c-9d31-49a1-bd74-62791d74db2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:02.765069 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-a6f4c69c-9d31-49a1-bd74-62791d74db2c None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:07:02.766332 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-a6f4c69c-9d31-49a1-bd74-62791d74db2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:07:02.817765 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc51a8d8-452d-4f1f-925e-87c6f60297b5 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:02.827345 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:02.861928 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:02.867455 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22b5256-4b30-4bee-9f7e-ecf4966cbc9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:02.868185 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22b5256-4b30-4bee-9f7e-ecf4966cbc9c None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:02.868474 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22b5256-4b30-4bee-9f7e-ecf4966cbc9c None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:02.868752 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22b5256-4b30-4bee-9f7e-ecf4966cbc9c None admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:02.869492 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22b5256-4b30-4bee-9f7e-ecf4966cbc9c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:07:02.869803 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22b5256-4b30-4bee-9f7e-ecf4966cbc9c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:02.871204 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22b5256-4b30-4bee-9f7e-ecf4966cbc9c None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:02.890006 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 353/707] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Jun 29 11:07:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:07:02.891300 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22b5256-4b30-4bee-9f7e-ecf4966cbc9c None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:02.902735 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 355/708] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:07:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 11:07:02.928782 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f4c69c-9d31-49a1-bd74-62791d74db2c None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:03.043962 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22b5256-4b30-4bee-9f7e-ecf4966cbc9c None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:03.072468 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f4c69c-9d31-49a1-bd74-62791d74db2c None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:03.080239 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45c2469-03ba-4fe8-93f1-7fd3e90c5fc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:03.081044 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45c2469-03ba-4fe8-93f1-7fd3e90c5fc0 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:03.081326 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45c2469-03ba-4fe8-93f1-7fd3e90c5fc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:03.081604 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45c2469-03ba-4fe8-93f1-7fd3e90c5fc0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:03.089465 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45c2469-03ba-4fe8-93f1-7fd3e90c5fc0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:03.090916 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45c2469-03ba-4fe8-93f1-7fd3e90c5fc0 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:03.105869 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-c45c2469-03ba-4fe8-93f1-7fd3e90c5fc0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 11:07:03.105869 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:07:03.105869 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:07:03.105869 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:07:03.105869 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:07:03.105869 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:07:03.105869 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:07:03.105869 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:07:03.105869 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:07:03.105869 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:07:03.105869 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:07:03.105869 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:07:03.105869 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 11:07:03.105869 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 11:07:03.105869 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 11:07:03.106836 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 11:07:03.106836 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:07:03.106836 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:07:03.106836 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:07:03.106836 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:07:03.106836 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:07:03.106836 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:07:03.106836 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:07:03.106836 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jun 29 11:07:03.106836 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:07:03.106836 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:07:03.106836 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:07:03.106836 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:07:03.106836 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:07:03.106836 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:07:03.106836 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:07:03.106836 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:07:03.107772 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 11:07:03.107772 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 11:07:03.107772 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 11:07:03.107772 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 11:07:03.107772 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 11:07:03.107772 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 11:07:03.107772 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 11:07:03.107772 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jun 29 11:07:03.107772 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 354/709] 10.222.0.44 () {56 vars in 1111 bytes} [Wed Jun 29 11:07:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:07:03.109774 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402a8640-68b2-4697-86d6-d94cc1843155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:03.110587 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402a8640-68b2-4697-86d6-d94cc1843155 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:03.110872 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402a8640-68b2-4697-86d6-d94cc1843155 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:03.114696 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402a8640-68b2-4697-86d6-d94cc1843155 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:03.114696 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402a8640-68b2-4697-86d6-d94cc1843155 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:03.114696 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402a8640-68b2-4697-86d6-d94cc1843155 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:03.115589 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45c2469-03ba-4fe8-93f1-7fd3e90c5fc0 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:03.128762 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-402a8640-68b2-4697-86d6-d94cc1843155 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 29 11:07:03.128762 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:07:03.128762 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 29 11:07:03.128762 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 29 11:07:03.128762 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 29 11:07:03.128762 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 29 11:07:03.128762 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 29 11:07:03.128762 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 11:07:03.128762 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 29 11:07:03.128762 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:07:03.128762 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 29 11:07:03.128762 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:07:03.128762 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:07:03.128762 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:07:03.128762 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:07:03.128762 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:07:03.129900 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:07:03.129900 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:07:03.129900 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:07:03.129900 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:07:03.129900 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:07:03.129900 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:07:03.129900 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:07:03.129900 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 29 11:07:03.129900 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 29 11:07:03.129900 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 29 11:07:03.129900 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 29 11:07:03.129900 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:07:03.129900 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:07:03.129900 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:07:03.129900 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:07:03.129900 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:07:03.130787 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:07:03.130787 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:07:03.130787 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jun 29 11:07:03.130787 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:07:03.130787 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:07:03.130787 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:07:03.130787 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:07:03.130787 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:07:03.130787 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:07:03.130787 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:07:03.130787 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:07:03.130787 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 29 11:07:03.130787 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 11:07:03.130787 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 29 11:07:03.130787 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:07:03.130787 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 356/710] 10.222.0.44 () {56 vars in 1129 bytes} [Wed Jun 29 11:07:02 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:07:03.137986 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402a8640-68b2-4697-86d6-d94cc1843155 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:03.156069 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:03.189521 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:03.242704 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45c2469-03ba-4fe8-93f1-7fd3e90c5fc0 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:03.277009 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402a8640-68b2-4697-86d6-d94cc1843155 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:03.288776 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9c786b-7b72-4206-909c-1bb77666f106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:03.290202 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9c786b-7b72-4206-909c-1bb77666f106 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:03.290953 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9c786b-7b72-4206-909c-1bb77666f106 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:03.291440 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9c786b-7b72-4206-909c-1bb77666f106 None admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:03.292752 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9c786b-7b72-4206-909c-1bb77666f106 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:07:03.293287 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9c786b-7b72-4206-909c-1bb77666f106 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:03.295003 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9c786b-7b72-4206-909c-1bb77666f106 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:03.313422 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 355/711] 10.222.0.44 () {56 vars in 1148 bytes} [Wed Jun 29 11:07:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 11:07:03.315989 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed930cae-eaa8-4788-a955-07ffda615197 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:03.316491 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed930cae-eaa8-4788-a955-07ffda615197 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:03.317150 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed930cae-eaa8-4788-a955-07ffda615197 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:03.318981 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 356/712] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:07:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:07:03.323877 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6a682d-ac76-4596-94e1-3143daec18bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:03.323877 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6a682d-ac76-4596-94e1-3143daec18bd None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:03.323877 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6a682d-ac76-4596-94e1-3143daec18bd None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:03.323877 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6a682d-ac76-4596-94e1-3143daec18bd None admin] PATH_INFO: `/v3/services` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:03.324319 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6a682d-ac76-4596-94e1-3143daec18bd None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:07:03.324319 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6a682d-ac76-4596-94e1-3143daec18bd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:03.324319 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6a682d-ac76-4596-94e1-3143daec18bd None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:03.324769 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed930cae-eaa8-4788-a955-07ffda615197 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:03.329578 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 357/713] 10.222.0.44 () {56 vars in 1124 bytes} [Wed Jun 29 11:07:03 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:07:03.331013 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf43fed-1f7d-4568-956b-9bddf0186a52 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:03.331373 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf43fed-1f7d-4568-956b-9bddf0186a52 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:03.331727 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf43fed-1f7d-4568-956b-9bddf0186a52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:03.350628 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-7cf43fed-1f7d-4568-956b-9bddf0186a52 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:07:03.352048 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-7cf43fed-1f7d-4568-956b-9bddf0186a52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:07:03.433740 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:03.450322 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed930cae-eaa8-4788-a955-07ffda615197 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:03.459074 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 358/714] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:07:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 11:07:03.460220 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf43fed-1f7d-4568-956b-9bddf0186a52 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:03.511537 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af2cbb4-7a69-4cc9-a93c-e2acea6c39be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:03.512237 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af2cbb4-7a69-4cc9-a93c-e2acea6c39be None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:03.512525 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af2cbb4-7a69-4cc9-a93c-e2acea6c39be None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:03.512804 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af2cbb4-7a69-4cc9-a93c-e2acea6c39be None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:03.513654 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af2cbb4-7a69-4cc9-a93c-e2acea6c39be None admin] RBAC: Adding query filter params (role.id=e95c2ce63e114cc19a9dda6ad21e794f, scope.project.id=ebe5dffa4bb8440286687daefb6c98f4, user.id=796aeeff16df4739bddec1f6dee61bfe) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:07:03.513972 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af2cbb4-7a69-4cc9-a93c-e2acea6c39be None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:03.515536 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af2cbb4-7a69-4cc9-a93c-e2acea6c39be None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:03.520383 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:03.522818 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 357/715] 10.222.0.44 () {56 vars in 1374 bytes} [Wed Jun 29 11:07:03 2022] GET /identity/v3/role_assignments?user.id=796aeeff16df4739bddec1f6dee61bfe&scope.project.id=ebe5dffa4bb8440286687daefb6c98f4&role.id=e95c2ce63e114cc19a9dda6ad21e794f => generated 259 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:07:03.523698 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af2cbb4-7a69-4cc9-a93c-e2acea6c39be None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:03.605931 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf43fed-1f7d-4568-956b-9bddf0186a52 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:03.640406 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af2cbb4-7a69-4cc9-a93c-e2acea6c39be None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:03.646403 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561c5d86-0d14-4c6e-a250-efd2adef2c43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:03.647389 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561c5d86-0d14-4c6e-a250-efd2adef2c43 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:03.647925 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561c5d86-0d14-4c6e-a250-efd2adef2c43 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:03.648545 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561c5d86-0d14-4c6e-a250-efd2adef2c43 None admin] PATH_INFO: `/v3/services` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:03.649453 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561c5d86-0d14-4c6e-a250-efd2adef2c43 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:07:03.649969 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561c5d86-0d14-4c6e-a250-efd2adef2c43 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:03.651750 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561c5d86-0d14-4c6e-a250-efd2adef2c43 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:03.658926 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 359/716] 10.222.0.44 () {56 vars in 1124 bytes} [Wed Jun 29 11:07:03 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:07:03.695218 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670bbc8a-2142-4772-aa7d-30d9dfa12628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:03.696871 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670bbc8a-2142-4772-aa7d-30d9dfa12628 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:03.697382 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670bbc8a-2142-4772-aa7d-30d9dfa12628 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:03.697913 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670bbc8a-2142-4772-aa7d-30d9dfa12628 None admin] PATH_INFO: `/v3/services/network` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:03.698914 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670bbc8a-2142-4772-aa7d-30d9dfa12628 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:03.700690 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670bbc8a-2142-4772-aa7d-30d9dfa12628 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:03.722208 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-670bbc8a-2142-4772-aa7d-30d9dfa12628 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 29 11:07:03.722208 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:07:03.722208 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 29 11:07:03.722208 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 29 11:07:03.722208 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 29 11:07:03.722208 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 29 11:07:03.722208 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 29 11:07:03.722208 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 11:07:03.722208 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 29 11:07:03.722208 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jun 29 11:07:03.722208 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 29 11:07:03.722208 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jun 29 11:07:03.722208 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:07:03.722208 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:07:03.722208 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:07:03.722208 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:07:03.722208 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:07:03.723346 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:07:03.723346 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:07:03.723346 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:07:03.723346 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:07:03.723346 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:07:03.723346 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:07:03.723346 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 29 11:07:03.723346 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 29 11:07:03.723346 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 29 11:07:03.723346 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 29 11:07:03.723346 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:07:03.723346 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:07:03.723346 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:07:03.723346 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:07:03.723346 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:07:03.723346 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:07:03.724281 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:07:03.724281 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jun 29 11:07:03.724281 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:07:03.724281 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:07:03.724281 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:07:03.724281 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:07:03.724281 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:07:03.724281 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:07:03.724281 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:07:03.724281 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:07:03.724281 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 29 11:07:03.724281 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 11:07:03.724281 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 29 11:07:03.724281 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jun 29 11:07:03.727564 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 358/717] 10.222.0.44 () {56 vars in 1111 bytes} [Wed Jun 29 11:07:03 2022] GET /identity/v3/services/network => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:07:03.733935 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561c5d86-0d14-4c6e-a250-efd2adef2c43 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:03.871749 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:03.878734 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561c5d86-0d14-4c6e-a250-efd2adef2c43 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:03.922225 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932d6d54-9ac4-4568-9f97-29f16915a5cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:03.922903 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932d6d54-9ac4-4568-9f97-29f16915a5cd None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:03.923182 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932d6d54-9ac4-4568-9f97-29f16915a5cd None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:03.923457 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932d6d54-9ac4-4568-9f97-29f16915a5cd None admin] PATH_INFO: `/v3/services` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:03.924194 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932d6d54-9ac4-4568-9f97-29f16915a5cd None admin] RBAC: Adding query filter params (name=network) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:07:03.924500 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932d6d54-9ac4-4568-9f97-29f16915a5cd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:03.925869 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932d6d54-9ac4-4568-9f97-29f16915a5cd None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:03.931450 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 360/718] 10.222.0.44 () {56 vars in 1112 bytes} [Wed Jun 29 11:07:03 2022] GET /identity/v3/services?name=network => generated 124 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:07:03.937565 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670bbc8a-2142-4772-aa7d-30d9dfa12628 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:03.967187 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:04.117434 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670bbc8a-2142-4772-aa7d-30d9dfa12628 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:04.165603 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c8c15c-8fa3-4401-b7a8-fa8461192589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:04.166197 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c8c15c-8fa3-4401-b7a8-fa8461192589 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:04.166480 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c8c15c-8fa3-4401-b7a8-fa8461192589 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:04.166758 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c8c15c-8fa3-4401-b7a8-fa8461192589 None admin] PATH_INFO: `/v3/services` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:04.167469 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c8c15c-8fa3-4401-b7a8-fa8461192589 None admin] RBAC: Adding query filter params (type=network) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:07:04.167858 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c8c15c-8fa3-4401-b7a8-fa8461192589 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:04.169246 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c8c15c-8fa3-4401-b7a8-fa8461192589 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:04.176548 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 359/719] 10.222.0.44 () {56 vars in 1112 bytes} [Wed Jun 29 11:07:03 2022] GET /identity/v3/services?type=network => generated 124 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:07:05.255137 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8e58ae-9fab-47cc-a537-a89af208c439 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:05.255689 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8e58ae-9fab-47cc-a537-a89af208c439 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:05.255689 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8e58ae-9fab-47cc-a537-a89af208c439 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:05.257302 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 361/720] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:07:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:07:05.262755 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773795fe-6bde-4541-a980-ed32b1abe439 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:05.265246 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773795fe-6bde-4541-a980-ed32b1abe439 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:05.265246 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773795fe-6bde-4541-a980-ed32b1abe439 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:05.284908 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-773795fe-6bde-4541-a980-ed32b1abe439 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:07:05.286145 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-773795fe-6bde-4541-a980-ed32b1abe439 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:07:05.382312 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 360/721] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:07:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 11:07:05.422914 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8e58ae-9fab-47cc-a537-a89af208c439 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:05.560581 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8e58ae-9fab-47cc-a537-a89af208c439 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:05.578392 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa371d9c-9073-445f-80eb-297e4bb3c0bf None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:05.578726 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa371d9c-9073-445f-80eb-297e4bb3c0bf None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:05.578970 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa371d9c-9073-445f-80eb-297e4bb3c0bf None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:05.581249 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 361/722] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:07:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:07:05.585897 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa43571-782b-4a61-a42b-bc379586164d None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:05.586254 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa43571-782b-4a61-a42b-bc379586164d None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:05.586536 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa43571-782b-4a61-a42b-bc379586164d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:05.603471 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ccd630-f842-40b5-8d64-2c9b8884d113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:05.604498 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ccd630-f842-40b5-8d64-2c9b8884d113 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:05.604951 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ccd630-f842-40b5-8d64-2c9b8884d113 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:05.605383 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ccd630-f842-40b5-8d64-2c9b8884d113 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:05.616760 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ccd630-f842-40b5-8d64-2c9b8884d113 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 11:07:05.620827 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-5aa43571-782b-4a61-a42b-bc379586164d None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:07:05.622028 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-5aa43571-782b-4a61-a42b-bc379586164d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:07:05.624990 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:05.625386 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-c1ccd630-f842-40b5-8d64-2c9b8884d113 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 29 11:07:05.625386 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:07:05.625386 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:07:05.625386 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:07:05.625386 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:07:05.625386 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:07:05.625386 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:07:05.625386 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:07:05.625386 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:07:05.625386 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:07:05.625386 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:07:05.625386 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:07:05.625386 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 11:07:05.625386 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 11:07:05.625386 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 11:07:05.625386 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 11:07:05.626309 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 11:07:05.626309 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 11:07:05.626309 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 11:07:05.626309 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 11:07:05.626309 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:07:05.626309 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:07:05.626309 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:07:05.626309 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:07:05.626309 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:07:05.626309 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:07:05.626309 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:07:05.626309 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jun 29 11:07:05.626309 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:07:05.626309 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:07:05.626309 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:07:05.626309 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:07:05.627195 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:07:05.627195 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:07:05.627195 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:07:05.627195 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:07:05.627195 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 11:07:05.627195 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 11:07:05.627195 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 11:07:05.627195 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 11:07:05.627195 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 11:07:05.627195 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 11:07:05.627195 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 29 11:07:05.627195 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:07:05.627195 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 362/723] 10.222.0.44 () {56 vars in 1102 bytes} [Wed Jun 29 11:07:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:07:05.641484 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ccd630-f842-40b5-8d64-2c9b8884d113 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:05.700672 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 362/724] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:07:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 11:07:05.701311 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:05.724273 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa43571-782b-4a61-a42b-bc379586164d None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:05.875007 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ccd630-f842-40b5-8d64-2c9b8884d113 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:05.915338 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b25aa99-4228-483f-83ec-4c90345c2605 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:05.916043 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b25aa99-4228-483f-83ec-4c90345c2605 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:05.916330 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b25aa99-4228-483f-83ec-4c90345c2605 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:05.916611 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b25aa99-4228-483f-83ec-4c90345c2605 None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:05.917790 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b25aa99-4228-483f-83ec-4c90345c2605 None admin] RBAC: Adding query filter params (name=service) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:07:05.918107 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b25aa99-4228-483f-83ec-4c90345c2605 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:05.919637 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b25aa99-4228-483f-83ec-4c90345c2605 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:05.929416 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 363/725] 10.222.0.44 () {56 vars in 1103 bytes} [Wed Jun 29 11:07:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:07:05.934845 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b25aa99-4228-483f-83ec-4c90345c2605 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:05.950064 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa43571-782b-4a61-a42b-bc379586164d None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:06.026572 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280aa6ab-38b7-4f85-987f-8d865f454da2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:06.027736 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280aa6ab-38b7-4f85-987f-8d865f454da2 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:06.032868 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280aa6ab-38b7-4f85-987f-8d865f454da2 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:06.033376 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280aa6ab-38b7-4f85-987f-8d865f454da2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:06.048200 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:06.056193 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:06.059395 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280aa6ab-38b7-4f85-987f-8d865f454da2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 11:07:06.074376 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-280aa6ab-38b7-4f85-987f-8d865f454da2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 11:07:06.074376 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:07:06.074376 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:07:06.074376 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:07:06.074376 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:07:06.074376 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:07:06.074376 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:07:06.074376 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:07:06.074376 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:07:06.074376 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:07:06.074376 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:07:06.074376 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:07:06.074376 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 11:07:06.074376 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 11:07:06.074376 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 11:07:06.074376 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 11:07:06.075608 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 11:07:06.075608 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 11:07:06.075608 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 11:07:06.075608 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 11:07:06.075608 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:07:06.075608 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:07:06.075608 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:07:06.075608 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:07:06.075608 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:07:06.075608 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:07:06.075608 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:07:06.075608 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jun 29 11:07:06.075608 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:07:06.075608 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:07:06.075608 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:07:06.075608 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:07:06.076530 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:07:06.076530 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:07:06.076530 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:07:06.076530 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:07:06.076530 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 11:07:06.076530 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 11:07:06.076530 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 11:07:06.076530 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 11:07:06.076530 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 11:07:06.076530 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 11:07:06.076530 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 11:07:06.076530 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jun 29 11:07:06.086593 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 363/726] 10.222.0.44 () {56 vars in 1096 bytes} [Wed Jun 29 11:07:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 363 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 11:07:06.090647 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280aa6ab-38b7-4f85-987f-8d865f454da2 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:06.098068 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b25aa99-4228-483f-83ec-4c90345c2605 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:06.181529 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4316bb65-d740-4e56-8bf2-f54b6936e900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:06.182304 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4316bb65-d740-4e56-8bf2-f54b6936e900 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:06.182594 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4316bb65-d740-4e56-8bf2-f54b6936e900 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:06.182874 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4316bb65-d740-4e56-8bf2-f54b6936e900 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:06.209004 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4316bb65-d740-4e56-8bf2-f54b6936e900 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:06.210516 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4316bb65-d740-4e56-8bf2-f54b6936e900 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:06.224639 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280aa6ab-38b7-4f85-987f-8d865f454da2 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:06.244188 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-4316bb65-d740-4e56-8bf2-f54b6936e900 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 11:07:06.244188 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:07:06.244188 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:07:06.244188 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:07:06.244188 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:07:06.244188 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:07:06.244188 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:07:06.244188 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:07:06.244188 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:07:06.244188 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:07:06.244188 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:07:06.244188 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:07:06.244188 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 11:07:06.244188 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 11:07:06.244188 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 11:07:06.245122 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 11:07:06.245122 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:07:06.245122 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:07:06.245122 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:07:06.245122 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:07:06.245122 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:07:06.245122 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:07:06.245122 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:07:06.245122 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jun 29 11:07:06.245122 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:07:06.245122 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:07:06.245122 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:07:06.245122 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:07:06.245122 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:07:06.245122 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:07:06.245122 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:07:06.245122 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:07:06.246025 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 11:07:06.246025 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 11:07:06.246025 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 11:07:06.246025 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:07:06.249127 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 364/727] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jun 29 11:07:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:07:06.250833 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a706536-5d9b-49e7-8971-fe5b1256be3e None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:06.251325 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a706536-5d9b-49e7-8971-fe5b1256be3e None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:06.251805 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a706536-5d9b-49e7-8971-fe5b1256be3e None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:06.253598 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 365/728] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:07:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:07:06.257278 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a706536-5d9b-49e7-8971-fe5b1256be3e None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:06.271167 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbed01c6-0725-4faa-9a19-5d0e526e4fcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:06.272676 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbed01c6-0725-4faa-9a19-5d0e526e4fcb None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:06.273113 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbed01c6-0725-4faa-9a19-5d0e526e4fcb None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:06.273537 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbed01c6-0725-4faa-9a19-5d0e526e4fcb None admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:06.274605 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbed01c6-0725-4faa-9a19-5d0e526e4fcb None admin] RBAC: Adding query filter params (name=admin) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:07:06.275471 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbed01c6-0725-4faa-9a19-5d0e526e4fcb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:06.277314 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbed01c6-0725-4faa-9a19-5d0e526e4fcb None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:06.289317 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 364/729] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Jun 29 11:07:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:07:06.290767 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ab20bb-3962-46dc-ae90-f53cde3cb56c None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:06.291194 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ab20bb-3962-46dc-ae90-f53cde3cb56c None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:06.291626 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ab20bb-3962-46dc-ae90-f53cde3cb56c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:06.312551 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-a8ab20bb-3962-46dc-ae90-f53cde3cb56c None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:07:06.314549 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-a8ab20bb-3962-46dc-ae90-f53cde3cb56c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:07:06.372518 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a706536-5d9b-49e7-8971-fe5b1256be3e None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:06.377642 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:06.388994 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 365/730] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:07:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 11:07:06.390201 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ab20bb-3962-46dc-ae90-f53cde3cb56c None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:06.414494 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:06.419784 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2730b3ec-855c-47ca-a68f-2a83c92b8c1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:06.420813 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2730b3ec-855c-47ca-a68f-2a83c92b8c1e None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:06.421456 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2730b3ec-855c-47ca-a68f-2a83c92b8c1e None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:06.424194 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2730b3ec-855c-47ca-a68f-2a83c92b8c1e None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:06.424194 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2730b3ec-855c-47ca-a68f-2a83c92b8c1e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:07:06.424194 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2730b3ec-855c-47ca-a68f-2a83c92b8c1e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:06.424835 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2730b3ec-855c-47ca-a68f-2a83c92b8c1e None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:06.437833 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 366/731] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Jun 29 11:07:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:07:06.439332 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be38b228-46b0-4c85-b527-47680d901dd8 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:06.439782 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be38b228-46b0-4c85-b527-47680d901dd8 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:06.440227 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be38b228-46b0-4c85-b527-47680d901dd8 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:06.442449 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 367/732] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:07:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:07:06.443496 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be38b228-46b0-4c85-b527-47680d901dd8 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:06.512084 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ab20bb-3962-46dc-ae90-f53cde3cb56c None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:06.551806 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd336ee-1dbd-4695-a036-037d63f5bc5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:06.552636 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd336ee-1dbd-4695-a036-037d63f5bc5a None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:06.552946 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd336ee-1dbd-4695-a036-037d63f5bc5a None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:06.553224 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd336ee-1dbd-4695-a036-037d63f5bc5a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:06.558011 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be38b228-46b0-4c85-b527-47680d901dd8 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:06.563404 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd336ee-1dbd-4695-a036-037d63f5bc5a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:06.564868 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd336ee-1dbd-4695-a036-037d63f5bc5a None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:06.586056 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-7cd336ee-1dbd-4695-a036-037d63f5bc5a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 11:07:06.586056 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:07:06.586056 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:07:06.586056 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:07:06.586056 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:07:06.586056 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:07:06.586056 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:07:06.586056 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:07:06.586056 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:07:06.586056 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:07:06.586056 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:07:06.586056 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:07:06.586056 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 11:07:06.586056 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 11:07:06.586056 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 11:07:06.587027 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 11:07:06.587027 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:07:06.587027 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:07:06.587027 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:07:06.587027 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:07:06.587027 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:07:06.587027 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:07:06.587027 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:07:06.587027 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jun 29 11:07:06.587027 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:07:06.587027 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:07:06.587027 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:07:06.587027 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:07:06.587027 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:07:06.587027 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:07:06.587027 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:07:06.587027 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:07:06.587985 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 11:07:06.587985 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 11:07:06.587985 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 11:07:06.587985 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jun 29 11:07:06.587985 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 366/733] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jun 29 11:07:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:07:06.588278 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd336ee-1dbd-4695-a036-037d63f5bc5a None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:06.598586 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb28386-1e3e-41c0-9b77-60ad60f5145d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:06.599251 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb28386-1e3e-41c0-9b77-60ad60f5145d None admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:06.599531 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb28386-1e3e-41c0-9b77-60ad60f5145d None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:06.599801 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb28386-1e3e-41c0-9b77-60ad60f5145d None admin] PATH_INFO: `/v3/services` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:06.600545 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb28386-1e3e-41c0-9b77-60ad60f5145d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:06.601910 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb28386-1e3e-41c0-9b77-60ad60f5145d None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:06.636790 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:06.691043 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd336ee-1dbd-4695-a036-037d63f5bc5a None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:06.705358 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 368/734] 10.222.0.44 () {60 vars in 1141 bytes} [Wed Jun 29 11:07:06 2022] POST /identity/v3/services => generated 251 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:07:06.705358 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4446edc6-5780-443d-9db5-f57d38449e5b None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:06.705358 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4446edc6-5780-443d-9db5-f57d38449e5b None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:06.705358 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4446edc6-5780-443d-9db5-f57d38449e5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:06.734550 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-4446edc6-5780-443d-9db5-f57d38449e5b None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:07:06.735933 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-4446edc6-5780-443d-9db5-f57d38449e5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:07:06.746662 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f784fdd4-e985-4adb-93fb-f2d0e66b6b9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:06.747483 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f784fdd4-e985-4adb-93fb-f2d0e66b6b9d None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:06.747762 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f784fdd4-e985-4adb-93fb-f2d0e66b6b9d None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:06.748177 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f784fdd4-e985-4adb-93fb-f2d0e66b6b9d None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:06.756903 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f784fdd4-e985-4adb-93fb-f2d0e66b6b9d None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:06.758509 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f784fdd4-e985-4adb-93fb-f2d0e66b6b9d None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:06.776196 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-f784fdd4-e985-4adb-93fb-f2d0e66b6b9d None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 29 11:07:06.776196 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:07:06.776196 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:07:06.776196 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:07:06.776196 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:07:06.776196 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:07:06.776196 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:07:06.776196 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:07:06.776196 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:07:06.776196 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:07:06.776196 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:07:06.776196 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:07:06.776196 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 11:07:06.776196 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 11:07:06.776196 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 11:07:06.776196 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 11:07:06.777241 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:07:06.777241 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:07:06.777241 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 11:07:06.777241 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 11:07:06.777241 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 11:07:06.777241 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 11:07:06.777241 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:07:06.777241 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:07:06.777241 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:07:06.777241 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:07:06.777241 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:07:06.777241 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jun 29 11:07:06.777241 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:07:06.777241 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:07:06.777241 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:07:06.777241 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:07:06.777241 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:07:06.777241 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:07:06.778204 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:07:06.778204 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:07:06.778204 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 11:07:06.778204 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 11:07:06.778204 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 11:07:06.778204 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 11:07:06.778204 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 11:07:06.778204 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 11:07:06.778204 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 29 11:07:06.778204 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jun 29 11:07:06.782475 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 367/735] 10.222.0.44 () {56 vars in 1117 bytes} [Wed Jun 29 11:07:06 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:07:06.783517 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f784fdd4-e985-4adb-93fb-f2d0e66b6b9d None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:06.795535 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:06.819383 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 369/736] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:07:06 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 11:07:06.820100 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4446edc6-5780-443d-9db5-f57d38449e5b None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:06.903082 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:06.908911 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f784fdd4-e985-4adb-93fb-f2d0e66b6b9d None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:06.940521 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4446edc6-5780-443d-9db5-f57d38449e5b None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:06.952119 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c70249e-c0fa-4e20-b617-ff1f5ca9e668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:06.952712 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c70249e-c0fa-4e20-b617-ff1f5ca9e668 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:06.952994 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c70249e-c0fa-4e20-b617-ff1f5ca9e668 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:06.953268 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c70249e-c0fa-4e20-b617-ff1f5ca9e668 None admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:06.953986 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c70249e-c0fa-4e20-b617-ff1f5ca9e668 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:07:06.954291 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c70249e-c0fa-4e20-b617-ff1f5ca9e668 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:06.955656 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c70249e-c0fa-4e20-b617-ff1f5ca9e668 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:06.968561 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 368/737] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Jun 29 11:07:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:07:06.970263 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdd7211-330b-46f9-89bd-c8e14b637ba8 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:06.970758 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdd7211-330b-46f9-89bd-c8e14b637ba8 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:06.971251 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdd7211-330b-46f9-89bd-c8e14b637ba8 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:06.973689 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 369/738] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:07:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:07:06.974683 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdd7211-330b-46f9-89bd-c8e14b637ba8 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:06.983556 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e6aeb3-1b24-49a2-b88a-a149b6dc8567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:06.984295 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e6aeb3-1b24-49a2-b88a-a149b6dc8567 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:06.984573 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e6aeb3-1b24-49a2-b88a-a149b6dc8567 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:06.984843 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e6aeb3-1b24-49a2-b88a-a149b6dc8567 None admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:06.985674 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e6aeb3-1b24-49a2-b88a-a149b6dc8567 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:07:06.985984 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e6aeb3-1b24-49a2-b88a-a149b6dc8567 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:06.987392 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e6aeb3-1b24-49a2-b88a-a149b6dc8567 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:07.005822 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 370/739] 10.222.0.44 () {56 vars in 1149 bytes} [Wed Jun 29 11:07:06 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:07:07.009223 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e6aeb3-1b24-49a2-b88a-a149b6dc8567 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:07.053318 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:07.081847 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdd7211-330b-46f9-89bd-c8e14b637ba8 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:07.113511 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e6aeb3-1b24-49a2-b88a-a149b6dc8567 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:07.117180 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244ada92-6739-4316-96a2-22cbaf431411 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:07.117974 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244ada92-6739-4316-96a2-22cbaf431411 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:07.118259 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244ada92-6739-4316-96a2-22cbaf431411 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:07.118535 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244ada92-6739-4316-96a2-22cbaf431411 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:07.129586 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244ada92-6739-4316-96a2-22cbaf431411 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 11:07:07.137304 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-244ada92-6739-4316-96a2-22cbaf431411 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 11:07:07.137304 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:07:07.137304 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:07:07.137304 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:07:07.137304 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:07:07.137304 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:07:07.137304 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:07:07.137304 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:07:07.137304 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:07:07.137304 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:07:07.137304 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:07:07.137304 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:07:07.137304 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 11:07:07.137304 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 11:07:07.137304 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 11:07:07.137304 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 11:07:07.138245 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 11:07:07.138245 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 11:07:07.138245 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 11:07:07.138245 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 11:07:07.138245 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:07:07.138245 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:07:07.138245 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:07:07.138245 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:07:07.138245 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:07:07.138245 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:07:07.138245 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:07:07.138245 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jun 29 11:07:07.138245 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:07:07.138245 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:07:07.138245 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:07:07.138245 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:07:07.139117 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:07:07.139117 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:07:07.139117 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:07:07.139117 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:07:07.139117 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 11:07:07.139117 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 11:07:07.139117 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 11:07:07.139117 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 11:07:07.139117 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 11:07:07.139117 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 11:07:07.139117 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 11:07:07.139117 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jun 29 11:07:07.143498 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 370/740] 10.222.0.44 () {56 vars in 1096 bytes} [Wed Jun 29 11:07:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:07:07.145310 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54e3dee-2c10-4dee-8139-5277eedec6db None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:07.145605 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54e3dee-2c10-4dee-8139-5277eedec6db None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:07.145884 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54e3dee-2c10-4dee-8139-5277eedec6db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:07.151621 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fcaa29-2f25-4f75-b91e-5496dec5d7c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:07.152440 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fcaa29-2f25-4f75-b91e-5496dec5d7c7 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:07.152720 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fcaa29-2f25-4f75-b91e-5496dec5d7c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:07.152990 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fcaa29-2f25-4f75-b91e-5496dec5d7c7 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:07.160133 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fcaa29-2f25-4f75-b91e-5496dec5d7c7 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:07.162105 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fcaa29-2f25-4f75-b91e-5496dec5d7c7 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:07.163144 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-f54e3dee-2c10-4dee-8139-5277eedec6db None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:07:07.164388 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-f54e3dee-2c10-4dee-8139-5277eedec6db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:07:07.176784 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-89fcaa29-2f25-4f75-b91e-5496dec5d7c7 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 29 11:07:07.176784 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:07:07.176784 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:07:07.176784 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:07:07.176784 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:07:07.176784 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:07:07.176784 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:07:07.176784 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:07:07.176784 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:07:07.176784 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:07:07.176784 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:07:07.176784 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:07:07.176784 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 11:07:07.176784 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 11:07:07.176784 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 11:07:07.176784 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 11:07:07.177732 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:07:07.177732 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:07:07.177732 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 11:07:07.177732 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 11:07:07.177732 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 11:07:07.177732 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 11:07:07.177732 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:07:07.177732 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:07:07.177732 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:07:07.177732 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:07:07.177732 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:07:07.177732 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jun 29 11:07:07.177732 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:07:07.177732 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:07:07.177732 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:07:07.177732 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:07:07.177732 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:07:07.177732 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:07:07.178672 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:07:07.178672 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:07:07.178672 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 11:07:07.178672 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 11:07:07.178672 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 11:07:07.178672 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 11:07:07.178672 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 11:07:07.178672 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 11:07:07.178672 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 29 11:07:07.178672 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:07:07.178672 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 371/741] 10.222.0.44 () {56 vars in 1096 bytes} [Wed Jun 29 11:07:07 2022] GET /identity/v3/users/swift => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:07:07.179314 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fcaa29-2f25-4f75-b91e-5496dec5d7c7 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:07.180482 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:07.243938 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 371/742] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:07:07 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 11:07:07.243938 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54e3dee-2c10-4dee-8139-5277eedec6db None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:07.294082 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:07.298083 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fcaa29-2f25-4f75-b91e-5496dec5d7c7 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:07.335402 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24adb67-e8ef-4e7e-8443-a7efa99f7b98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:07.336602 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24adb67-e8ef-4e7e-8443-a7efa99f7b98 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:07.337154 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24adb67-e8ef-4e7e-8443-a7efa99f7b98 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:07.337578 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24adb67-e8ef-4e7e-8443-a7efa99f7b98 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:07.339700 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54e3dee-2c10-4dee-8139-5277eedec6db None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:07.376775 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c562cabe-79b6-4ea5-97f2-ecdef367906e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:07.377480 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c562cabe-79b6-4ea5-97f2-ecdef367906e None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:07.377758 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c562cabe-79b6-4ea5-97f2-ecdef367906e None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:07.378029 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c562cabe-79b6-4ea5-97f2-ecdef367906e None admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:07.378766 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c562cabe-79b6-4ea5-97f2-ecdef367906e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:07:07.379106 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c562cabe-79b6-4ea5-97f2-ecdef367906e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:07.380502 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c562cabe-79b6-4ea5-97f2-ecdef367906e None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:07.388642 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 372/743] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Jun 29 11:07:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:07:07.389243 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c562cabe-79b6-4ea5-97f2-ecdef367906e None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:07.428017 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:07.454960 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24adb67-e8ef-4e7e-8443-a7efa99f7b98 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:07.456752 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24adb67-e8ef-4e7e-8443-a7efa99f7b98 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:07.515640 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c562cabe-79b6-4ea5-97f2-ecdef367906e None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:07.551706 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde21aa5-c6f1-41cd-8a04-ad59420a63ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:07.552398 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde21aa5-c6f1-41cd-8a04-ad59420a63ba None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:07.552683 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde21aa5-c6f1-41cd-8a04-ad59420a63ba None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:07.552961 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde21aa5-c6f1-41cd-8a04-ad59420a63ba None admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:07.553794 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde21aa5-c6f1-41cd-8a04-ad59420a63ba None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:07:07.554102 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde21aa5-c6f1-41cd-8a04-ad59420a63ba None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:07.555528 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde21aa5-c6f1-41cd-8a04-ad59420a63ba None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:07.561646 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:07.569832 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 372/744] 10.222.0.44 () {58 vars in 1282 bytes} [Wed Jun 29 11:07:07 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 29 11:07:07.569832 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24adb67-e8ef-4e7e-8443-a7efa99f7b98 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:07.574275 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 373/745] 10.222.0.44 () {56 vars in 1135 bytes} [Wed Jun 29 11:07:07 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:07:07.578700 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde21aa5-c6f1-41cd-8a04-ad59420a63ba None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:07.648402 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:07.684062 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde21aa5-c6f1-41cd-8a04-ad59420a63ba None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:07.684976 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24adb67-e8ef-4e7e-8443-a7efa99f7b98 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:07.720267 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f987e0-710e-4894-a8c7-874821d2c391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:07.721047 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f987e0-710e-4894-a8c7-874821d2c391 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:07.721264 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f39a33-8d7f-4ab7-bd74-f1c39ee5fa2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:07.721392 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f987e0-710e-4894-a8c7-874821d2c391 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:07.721582 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f987e0-710e-4894-a8c7-874821d2c391 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:07.721922 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f39a33-8d7f-4ab7-bd74-f1c39ee5fa2e None admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:07.722204 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f39a33-8d7f-4ab7-bd74-f1c39ee5fa2e None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:07.722478 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f39a33-8d7f-4ab7-bd74-f1c39ee5fa2e None admin] PATH_INFO: `/v3/services` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:07.723246 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f39a33-8d7f-4ab7-bd74-f1c39ee5fa2e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:07.724717 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f39a33-8d7f-4ab7-bd74-f1c39ee5fa2e None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:07.732119 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f987e0-710e-4894-a8c7-874821d2c391 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:07.733548 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f987e0-710e-4894-a8c7-874821d2c391 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:07.751676 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-a9f987e0-710e-4894-a8c7-874821d2c391 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 11:07:07.751676 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:07:07.751676 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:07:07.751676 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:07:07.751676 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:07:07.751676 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:07:07.751676 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:07:07.751676 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:07:07.751676 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:07:07.751676 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:07:07.751676 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:07:07.751676 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:07:07.751676 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 11:07:07.751676 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 11:07:07.751676 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 11:07:07.752711 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 11:07:07.752711 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:07:07.752711 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:07:07.752711 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:07:07.752711 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:07:07.752711 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:07:07.752711 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:07:07.752711 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:07:07.752711 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jun 29 11:07:07.752711 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:07:07.752711 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:07:07.752711 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:07:07.752711 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:07:07.752711 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:07:07.752711 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:07:07.752711 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:07:07.752711 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:07:07.753619 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 11:07:07.753619 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 11:07:07.753619 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 11:07:07.753619 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jun 29 11:07:07.753619 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 374/746] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jun 29 11:07:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:07:07.753619 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f987e0-710e-4894-a8c7-874821d2c391 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:07.803946 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 373/747] 10.222.0.44 () {60 vars in 1141 bytes} [Wed Jun 29 11:07:07 2022] POST /identity/v3/services => generated 240 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:07:07.803946 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f39a33-8d7f-4ab7-bd74-f1c39ee5fa2e None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:07.877872 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f987e0-710e-4894-a8c7-874821d2c391 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:07.889578 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:07.904689 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:07.931435 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c09cd3-1d63-42a8-a806-f9794408d956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:07.932569 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c09cd3-1d63-42a8-a806-f9794408d956 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:07.932857 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c09cd3-1d63-42a8-a806-f9794408d956 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:07.933135 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c09cd3-1d63-42a8-a806-f9794408d956 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:07.944739 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c09cd3-1d63-42a8-a806-f9794408d956 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:07.946216 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c09cd3-1d63-42a8-a806-f9794408d956 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:07.952638 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f39a33-8d7f-4ab7-bd74-f1c39ee5fa2e None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:07.966373 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-73c09cd3-1d63-42a8-a806-f9794408d956 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 11:07:07.966373 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:07:07.966373 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:07:07.966373 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:07:07.966373 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:07:07.966373 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:07:07.966373 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:07:07.966373 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:07:07.966373 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:07:07.966373 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:07:07.966373 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:07:07.966373 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:07:07.966373 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 11:07:07.966373 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 11:07:07.966373 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 11:07:07.967363 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 11:07:07.967363 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:07:07.967363 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:07:07.967363 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:07:07.967363 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:07:07.967363 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:07:07.967363 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:07:07.967363 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:07:07.967363 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jun 29 11:07:07.967363 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:07:07.967363 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:07:07.967363 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:07:07.967363 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:07:07.967363 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:07:07.967363 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:07:07.967363 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:07:07.967363 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:07:07.968317 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 11:07:07.968317 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 11:07:07.968317 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 11:07:07.968317 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jun 29 11:07:07.972360 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 375/748] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jun 29 11:07:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:07:07.973392 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c09cd3-1d63-42a8-a806-f9794408d956 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:08.003582 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d80ffb3-38bf-4dfb-9876-e90de8b92062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:08.004435 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d80ffb3-38bf-4dfb-9876-e90de8b92062 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:08.004788 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d80ffb3-38bf-4dfb-9876-e90de8b92062 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:08.005142 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d80ffb3-38bf-4dfb-9876-e90de8b92062 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:08.095524 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d80ffb3-38bf-4dfb-9876-e90de8b92062 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:08.097339 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d80ffb3-38bf-4dfb-9876-e90de8b92062 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:08.119326 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c09cd3-1d63-42a8-a806-f9794408d956 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:08.159650 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d83416-8d48-43dc-baa3-5fcf88f3f390 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:08.161219 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d83416-8d48-43dc-baa3-5fcf88f3f390 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:08.161704 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d83416-8d48-43dc-baa3-5fcf88f3f390 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:08.162208 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d83416-8d48-43dc-baa3-5fcf88f3f390 None admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:08.163126 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d83416-8d48-43dc-baa3-5fcf88f3f390 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:07:08.163630 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d83416-8d48-43dc-baa3-5fcf88f3f390 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:08.165219 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d83416-8d48-43dc-baa3-5fcf88f3f390 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:08.166310 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:08.179100 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 376/749] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Jun 29 11:07:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:07:08.180307 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d83416-8d48-43dc-baa3-5fcf88f3f390 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:08.213783 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:08.228362 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 374/750] 10.222.0.44 () {58 vars in 1282 bytes} [Wed Jun 29 11:07:07 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 29 11:07:08.229311 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d80ffb3-38bf-4dfb-9876-e90de8b92062 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:08.294320 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d83416-8d48-43dc-baa3-5fcf88f3f390 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:08.333687 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1496f5-2def-4aa6-b289-046498c23613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:08.334774 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1496f5-2def-4aa6-b289-046498c23613 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:08.335261 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1496f5-2def-4aa6-b289-046498c23613 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:08.335756 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1496f5-2def-4aa6-b289-046498c23613 None admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:08.336751 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1496f5-2def-4aa6-b289-046498c23613 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:07:08.337393 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1496f5-2def-4aa6-b289-046498c23613 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:08.338978 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1496f5-2def-4aa6-b289-046498c23613 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:08.353240 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 377/751] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Jun 29 11:07:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:07:08.354934 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1496f5-2def-4aa6-b289-046498c23613 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:08.372659 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d80ffb3-38bf-4dfb-9876-e90de8b92062 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:08.409160 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e03fa5-7f58-4845-82d4-5f4ac0fe1895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:08.410063 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e03fa5-7f58-4845-82d4-5f4ac0fe1895 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:08.410419 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e03fa5-7f58-4845-82d4-5f4ac0fe1895 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:08.410832 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e03fa5-7f58-4845-82d4-5f4ac0fe1895 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:08.418827 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e03fa5-7f58-4845-82d4-5f4ac0fe1895 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:08.420559 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e03fa5-7f58-4845-82d4-5f4ac0fe1895 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:08.436594 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-72e03fa5-7f58-4845-82d4-5f4ac0fe1895 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 29 11:07:08.436594 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:07:08.436594 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:07:08.436594 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:07:08.436594 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:07:08.436594 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:07:08.436594 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:07:08.436594 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:07:08.436594 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:07:08.436594 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:07:08.436594 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:07:08.436594 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:07:08.436594 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 11:07:08.436594 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 11:07:08.436594 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 11:07:08.436594 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 11:07:08.437574 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:07:08.437574 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:07:08.437574 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 11:07:08.437574 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 11:07:08.437574 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 11:07:08.437574 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 11:07:08.437574 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:07:08.437574 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:07:08.437574 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:07:08.437574 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:07:08.437574 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:07:08.437574 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jun 29 11:07:08.437574 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:07:08.437574 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:07:08.437574 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:07:08.437574 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:07:08.437574 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:07:08.437574 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:07:08.438516 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:07:08.438516 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:07:08.438516 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 11:07:08.438516 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 11:07:08.438516 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 11:07:08.438516 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 11:07:08.438516 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 11:07:08.438516 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 11:07:08.438516 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 29 11:07:08.438516 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:07:08.438516 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 375/752] 10.222.0.44 () {56 vars in 1093 bytes} [Wed Jun 29 11:07:08 2022] GET /identity/v3/users/nova => generated 82 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:07:08.439113 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e03fa5-7f58-4845-82d4-5f4ac0fe1895 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:08.459822 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:08.464623 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:08.473520 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1496f5-2def-4aa6-b289-046498c23613 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:08.510443 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279244c7-9f72-46f3-9889-3d5086ff7210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:08.511475 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279244c7-9f72-46f3-9889-3d5086ff7210 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:08.511961 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279244c7-9f72-46f3-9889-3d5086ff7210 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:08.512536 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279244c7-9f72-46f3-9889-3d5086ff7210 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:08.525833 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279244c7-9f72-46f3-9889-3d5086ff7210 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:08.528130 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279244c7-9f72-46f3-9889-3d5086ff7210 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:08.548973 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e03fa5-7f58-4845-82d4-5f4ac0fe1895 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:08.553637 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-279244c7-9f72-46f3-9889-3d5086ff7210 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 11:07:08.553637 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:07:08.553637 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:07:08.553637 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:07:08.553637 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:07:08.553637 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:07:08.553637 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:07:08.553637 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:07:08.553637 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:07:08.553637 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:07:08.553637 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:07:08.553637 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:07:08.553637 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 11:07:08.553637 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 11:07:08.553637 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 11:07:08.554694 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 11:07:08.554694 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:07:08.554694 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:07:08.554694 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:07:08.554694 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:07:08.554694 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:07:08.554694 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:07:08.554694 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:07:08.554694 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jun 29 11:07:08.554694 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:07:08.554694 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:07:08.554694 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:07:08.554694 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:07:08.554694 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:07:08.554694 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:07:08.554694 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:07:08.554694 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:07:08.555788 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 11:07:08.555788 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 11:07:08.555788 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 11:07:08.555788 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jun 29 11:07:08.558249 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 378/753] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jun 29 11:07:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:07:08.559641 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279244c7-9f72-46f3-9889-3d5086ff7210 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:08.597353 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcc3707-2fc8-45a2-b71b-3b0a712bdfc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:08.598174 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcc3707-2fc8-45a2-b71b-3b0a712bdfc0 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:08.598453 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcc3707-2fc8-45a2-b71b-3b0a712bdfc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:08.598724 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcc3707-2fc8-45a2-b71b-3b0a712bdfc0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:08.606474 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcc3707-2fc8-45a2-b71b-3b0a712bdfc0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:08.607857 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcc3707-2fc8-45a2-b71b-3b0a712bdfc0 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:08.624120 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-2dcc3707-2fc8-45a2-b71b-3b0a712bdfc0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 11:07:08.624120 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:07:08.624120 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:07:08.624120 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:07:08.624120 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:07:08.624120 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:07:08.624120 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:07:08.624120 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:07:08.624120 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:07:08.624120 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:07:08.624120 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:07:08.624120 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:07:08.624120 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 11:07:08.624120 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 11:07:08.624120 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 11:07:08.625268 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 11:07:08.625268 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:07:08.625268 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:07:08.625268 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:07:08.625268 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:07:08.625268 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:07:08.625268 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:07:08.625268 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:07:08.625268 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jun 29 11:07:08.625268 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:07:08.625268 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:07:08.625268 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:07:08.625268 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:07:08.625268 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:07:08.625268 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:07:08.625268 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:07:08.625268 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:07:08.626412 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 11:07:08.626412 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 11:07:08.626412 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 11:07:08.626412 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 11:07:08.626412 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 11:07:08.626412 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 11:07:08.626412 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 11:07:08.626412 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:07:08.630777 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 376/754] 10.222.0.44 () {56 vars in 1111 bytes} [Wed Jun 29 11:07:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:07:08.631520 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcc3707-2fc8-45a2-b71b-3b0a712bdfc0 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:08.689827 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279244c7-9f72-46f3-9889-3d5086ff7210 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:08.726328 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6730127e-7831-4fdb-84be-1a2d0229cb47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:08.727077 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:08.727775 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6730127e-7831-4fdb-84be-1a2d0229cb47 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:08.728553 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6730127e-7831-4fdb-84be-1a2d0229cb47 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:08.729089 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6730127e-7831-4fdb-84be-1a2d0229cb47 None admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:08.730097 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6730127e-7831-4fdb-84be-1a2d0229cb47 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:07:08.730554 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6730127e-7831-4fdb-84be-1a2d0229cb47 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:08.732169 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6730127e-7831-4fdb-84be-1a2d0229cb47 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:08.745137 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:08.749678 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcc3707-2fc8-45a2-b71b-3b0a712bdfc0 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:08.755041 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 379/755] 10.222.0.44 () {56 vars in 1133 bytes} [Wed Jun 29 11:07:08 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:07:08.756160 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6730127e-7831-4fdb-84be-1a2d0229cb47 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:08.788983 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b3e6c8-e68a-42f3-a8d8-bffca4975da1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:08.789705 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b3e6c8-e68a-42f3-a8d8-bffca4975da1 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:08.789987 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b3e6c8-e68a-42f3-a8d8-bffca4975da1 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:08.790286 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b3e6c8-e68a-42f3-a8d8-bffca4975da1 None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:08.792156 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b3e6c8-e68a-42f3-a8d8-bffca4975da1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:07:08.792467 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b3e6c8-e68a-42f3-a8d8-bffca4975da1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:08.793819 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b3e6c8-e68a-42f3-a8d8-bffca4975da1 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:08.810770 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 377/756] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Jun 29 11:07:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:07:08.811912 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b3e6c8-e68a-42f3-a8d8-bffca4975da1 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:08.864554 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6730127e-7831-4fdb-84be-1a2d0229cb47 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:08.901578 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b247da-47cd-4f68-957b-45cccc00ad51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:08.902419 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b247da-47cd-4f68-957b-45cccc00ad51 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:08.902710 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b247da-47cd-4f68-957b-45cccc00ad51 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:08.902989 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b247da-47cd-4f68-957b-45cccc00ad51 None admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:08.903781 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b247da-47cd-4f68-957b-45cccc00ad51 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:07:08.904123 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b247da-47cd-4f68-957b-45cccc00ad51 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:08.905632 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b247da-47cd-4f68-957b-45cccc00ad51 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:08.920694 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 380/757] 10.222.0.44 () {56 vars in 1148 bytes} [Wed Jun 29 11:07:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:07:08.921912 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b247da-47cd-4f68-957b-45cccc00ad51 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:08.923677 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b3e6c8-e68a-42f3-a8d8-bffca4975da1 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:08.960624 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf72604-84e3-4008-af9c-4739ede84c24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:08.961416 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf72604-84e3-4008-af9c-4739ede84c24 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:08.961699 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf72604-84e3-4008-af9c-4739ede84c24 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:08.961974 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf72604-84e3-4008-af9c-4739ede84c24 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:08.970630 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:08.973789 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf72604-84e3-4008-af9c-4739ede84c24 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:08.975223 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf72604-84e3-4008-af9c-4739ede84c24 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:08.997954 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-adf72604-84e3-4008-af9c-4739ede84c24 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 11:07:08.997954 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:07:08.997954 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:07:08.997954 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:07:08.997954 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:07:08.997954 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:07:08.997954 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:07:08.997954 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:07:08.997954 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:07:08.997954 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:07:08.997954 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:07:08.997954 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:07:08.997954 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 11:07:08.997954 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 11:07:08.997954 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 11:07:08.999121 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 11:07:08.999121 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:07:08.999121 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:07:08.999121 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:07:08.999121 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:07:08.999121 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:07:08.999121 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:07:08.999121 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:07:08.999121 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jun 29 11:07:08.999121 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:07:08.999121 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:07:08.999121 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:07:08.999121 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:07:08.999121 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:07:08.999121 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:07:08.999121 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:07:08.999121 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:07:09.000082 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 11:07:09.000082 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 11:07:09.000082 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 11:07:09.000082 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:07:09.000082 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 378/758] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jun 29 11:07:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:07:09.000868 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcba4bd-192d-41bd-a343-538f31cc6e73 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:09.001163 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcba4bd-192d-41bd-a343-538f31cc6e73 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:09.001436 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcba4bd-192d-41bd-a343-538f31cc6e73 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:09.009240 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:09.009447 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 379/759] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:07:08 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:07:09.010607 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcba4bd-192d-41bd-a343-538f31cc6e73 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:09.063369 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b247da-47cd-4f68-957b-45cccc00ad51 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:09.100648 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd259ca-34d9-4788-ae14-941916c1701d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:09.101440 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd259ca-34d9-4788-ae14-941916c1701d None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:09.101724 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd259ca-34d9-4788-ae14-941916c1701d None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:09.102000 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd259ca-34d9-4788-ae14-941916c1701d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:09.108943 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd259ca-34d9-4788-ae14-941916c1701d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:09.110382 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd259ca-34d9-4788-ae14-941916c1701d None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:09.124941 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-4dd259ca-34d9-4788-ae14-941916c1701d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 11:07:09.124941 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:07:09.124941 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:07:09.124941 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:07:09.124941 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:07:09.124941 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:07:09.124941 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:07:09.124941 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:07:09.124941 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:07:09.124941 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:07:09.124941 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:07:09.124941 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:07:09.124941 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 11:07:09.124941 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 11:07:09.124941 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 11:07:09.125818 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 11:07:09.125818 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:07:09.125818 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:07:09.125818 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:07:09.125818 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:07:09.125818 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:07:09.125818 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:07:09.125818 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:07:09.125818 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jun 29 11:07:09.125818 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:07:09.125818 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:07:09.125818 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:07:09.125818 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:07:09.125818 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:07:09.125818 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:07:09.125818 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:07:09.125818 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:07:09.126720 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 11:07:09.126720 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 11:07:09.126720 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 11:07:09.126720 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 11:07:09.126720 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 11:07:09.126720 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 11:07:09.126720 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 11:07:09.126720 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jun 29 11:07:09.126720 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 381/760] 10.222.0.44 () {56 vars in 1111 bytes} [Wed Jun 29 11:07:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:07:09.127210 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd259ca-34d9-4788-ae14-941916c1701d None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:09.136022 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcba4bd-192d-41bd-a343-538f31cc6e73 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:09.171481 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4765c08-fb87-4204-beed-f3eac493313f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:09.172166 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4765c08-fb87-4204-beed-f3eac493313f None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:09.172449 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4765c08-fb87-4204-beed-f3eac493313f None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:09.172747 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4765c08-fb87-4204-beed-f3eac493313f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:09.173562 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4765c08-fb87-4204-beed-f3eac493313f None admin] RBAC: Adding query filter params (role.id=a8212be3426741cbb1b38b1646c66405, scope.project.id=ebe5dffa4bb8440286687daefb6c98f4, user.id=55487c2107a44c15b656b4865ee0d4fa) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:07:09.173873 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4765c08-fb87-4204-beed-f3eac493313f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:09.175276 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4765c08-fb87-4204-beed-f3eac493313f None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:09.180712 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 380/761] 10.222.0.44 () {56 vars in 1374 bytes} [Wed Jun 29 11:07:09 2022] GET /identity/v3/role_assignments?user.id=55487c2107a44c15b656b4865ee0d4fa&scope.project.id=ebe5dffa4bb8440286687daefb6c98f4&role.id=a8212be3426741cbb1b38b1646c66405 => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:07:09.181853 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1313aac0-d539-4875-85b7-db37987bd5f2 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:09.182132 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1313aac0-d539-4875-85b7-db37987bd5f2 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:09.182405 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1313aac0-d539-4875-85b7-db37987bd5f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:09.206651 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-1313aac0-d539-4875-85b7-db37987bd5f2 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:07:09.209136 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-1313aac0-d539-4875-85b7-db37987bd5f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:07:09.256069 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd259ca-34d9-4788-ae14-941916c1701d None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:09.276608 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:09.282256 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:09.300269 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 381/762] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:07:09 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 11:07:09.301876 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1313aac0-d539-4875-85b7-db37987bd5f2 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:09.303297 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521cd62f-e758-4789-a9a1-4b4e0e1ab96c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:09.304015 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521cd62f-e758-4789-a9a1-4b4e0e1ab96c None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:09.304297 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521cd62f-e758-4789-a9a1-4b4e0e1ab96c None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:09.304569 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521cd62f-e758-4789-a9a1-4b4e0e1ab96c None admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:09.309382 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521cd62f-e758-4789-a9a1-4b4e0e1ab96c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:07:09.309382 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521cd62f-e758-4789-a9a1-4b4e0e1ab96c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:09.309382 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521cd62f-e758-4789-a9a1-4b4e0e1ab96c None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:09.327922 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 382/763] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Jun 29 11:07:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:07:09.331207 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521cd62f-e758-4789-a9a1-4b4e0e1ab96c None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:09.435481 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1313aac0-d539-4875-85b7-db37987bd5f2 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:09.462682 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521cd62f-e758-4789-a9a1-4b4e0e1ab96c None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:09.476892 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2251e5bd-9c9f-48e5-aee8-9d6572d2cf4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:09.477992 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2251e5bd-9c9f-48e5-aee8-9d6572d2cf4c None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:09.478488 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2251e5bd-9c9f-48e5-aee8-9d6572d2cf4c None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:09.478906 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2251e5bd-9c9f-48e5-aee8-9d6572d2cf4c None admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:09.479856 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2251e5bd-9c9f-48e5-aee8-9d6572d2cf4c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:07:09.480314 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2251e5bd-9c9f-48e5-aee8-9d6572d2cf4c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:09.481911 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2251e5bd-9c9f-48e5-aee8-9d6572d2cf4c None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:09.494428 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 382/764] 10.222.0.44 () {56 vars in 1148 bytes} [Wed Jun 29 11:07:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:07:09.495515 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2251e5bd-9c9f-48e5-aee8-9d6572d2cf4c None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:09.497803 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9236243a-86eb-41d3-b7f8-0a3e27e989dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:09.498705 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9236243a-86eb-41d3-b7f8-0a3e27e989dc None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:09.498869 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9236243a-86eb-41d3-b7f8-0a3e27e989dc None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:09.499141 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9236243a-86eb-41d3-b7f8-0a3e27e989dc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:09.507439 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9236243a-86eb-41d3-b7f8-0a3e27e989dc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:09.508925 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9236243a-86eb-41d3-b7f8-0a3e27e989dc None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:09.524165 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-9236243a-86eb-41d3-b7f8-0a3e27e989dc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 11:07:09.524165 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:07:09.524165 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:07:09.524165 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:07:09.524165 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:07:09.524165 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:07:09.524165 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:07:09.524165 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:07:09.524165 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:07:09.524165 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:07:09.524165 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:07:09.524165 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:07:09.524165 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 11:07:09.524165 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 11:07:09.524165 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 11:07:09.525303 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 11:07:09.525303 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:07:09.525303 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:07:09.525303 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:07:09.525303 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:07:09.525303 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:07:09.525303 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:07:09.525303 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:07:09.525303 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jun 29 11:07:09.525303 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:07:09.525303 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:07:09.525303 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:07:09.525303 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:07:09.525303 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:07:09.525303 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:07:09.525303 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:07:09.525303 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:07:09.526368 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 11:07:09.526368 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 11:07:09.526368 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 11:07:09.526368 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 11:07:09.526368 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 11:07:09.526368 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 11:07:09.526368 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 11:07:09.526368 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jun 29 11:07:09.530169 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 383/765] 10.222.0.44 () {56 vars in 1111 bytes} [Wed Jun 29 11:07:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 11:07:09.531076 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9236243a-86eb-41d3-b7f8-0a3e27e989dc None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:09.550053 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:09.577446 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:09.609659 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2251e5bd-9c9f-48e5-aee8-9d6572d2cf4c None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:09.637351 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9236243a-86eb-41d3-b7f8-0a3e27e989dc None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:09.650046 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8def3f2c-e18d-4295-ba9d-b9917656c9ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:09.651166 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8def3f2c-e18d-4295-ba9d-b9917656c9ed None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:09.651600 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8def3f2c-e18d-4295-ba9d-b9917656c9ed None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:09.652142 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8def3f2c-e18d-4295-ba9d-b9917656c9ed None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:09.652929 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8def3f2c-e18d-4295-ba9d-b9917656c9ed None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:09.654524 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8def3f2c-e18d-4295-ba9d-b9917656c9ed None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:09.669227 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-8def3f2c-e18d-4295-ba9d-b9917656c9ed None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 29 11:07:09.669227 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:07:09.669227 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 29 11:07:09.669227 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 29 11:07:09.669227 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 29 11:07:09.669227 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 29 11:07:09.669227 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 29 11:07:09.669227 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 11:07:09.669227 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 29 11:07:09.669227 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:07:09.669227 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 29 11:07:09.669227 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:07:09.669227 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:07:09.669227 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:07:09.669227 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:07:09.669227 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:07:09.670443 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:07:09.670443 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:07:09.670443 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:07:09.670443 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:07:09.670443 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:07:09.670443 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:07:09.670443 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:07:09.670443 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 29 11:07:09.670443 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 29 11:07:09.670443 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 29 11:07:09.670443 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 29 11:07:09.670443 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:07:09.670443 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:07:09.670443 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:07:09.670443 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:07:09.670443 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:07:09.671465 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:07:09.671465 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:07:09.671465 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jun 29 11:07:09.671465 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:07:09.671465 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:07:09.671465 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:07:09.671465 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:07:09.671465 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:07:09.671465 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:07:09.671465 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:07:09.671465 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:07:09.671465 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 29 11:07:09.671465 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 11:07:09.671465 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 29 11:07:09.671465 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:07:09.679042 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 383/766] 10.222.0.44 () {56 vars in 1129 bytes} [Wed Jun 29 11:07:09 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:07:09.679042 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8def3f2c-e18d-4295-ba9d-b9917656c9ed None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:09.684428 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e95e3f5-2d73-4b3c-9513-9eba36ca2b6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:09.685266 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e95e3f5-2d73-4b3c-9513-9eba36ca2b6d None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:09.685708 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e95e3f5-2d73-4b3c-9513-9eba36ca2b6d None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:09.686052 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e95e3f5-2d73-4b3c-9513-9eba36ca2b6d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:09.687501 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e95e3f5-2d73-4b3c-9513-9eba36ca2b6d None admin] RBAC: Adding query filter params (role.id=e95c2ce63e114cc19a9dda6ad21e794f, scope.project.id=ebe5dffa4bb8440286687daefb6c98f4, user.id=e3b38493ba904759badc800e396befda) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:07:09.687501 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e95e3f5-2d73-4b3c-9513-9eba36ca2b6d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:09.688987 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e95e3f5-2d73-4b3c-9513-9eba36ca2b6d None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:09.696653 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 384/767] 10.222.0.44 () {56 vars in 1374 bytes} [Wed Jun 29 11:07:09 2022] GET /identity/v3/role_assignments?user.id=e3b38493ba904759badc800e396befda&scope.project.id=ebe5dffa4bb8440286687daefb6c98f4&role.id=e95c2ce63e114cc19a9dda6ad21e794f => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:07:09.697621 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e95e3f5-2d73-4b3c-9513-9eba36ca2b6d None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:09.798404 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8def3f2c-e18d-4295-ba9d-b9917656c9ed None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:09.805858 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e95e3f5-2d73-4b3c-9513-9eba36ca2b6d None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:09.807934 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:09.812839 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:09.839878 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a706a761-3c78-405a-a324-a62c936f9216 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:09.841089 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a706a761-3c78-405a-a324-a62c936f9216 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:09.841570 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a706a761-3c78-405a-a324-a62c936f9216 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:09.842048 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a706a761-3c78-405a-a324-a62c936f9216 None admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:09.843161 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a706a761-3c78-405a-a324-a62c936f9216 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:07:09.843706 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a706a761-3c78-405a-a324-a62c936f9216 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:09.845129 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6c0dd8-1abd-4f66-a5e4-a1d1931594ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:09.845645 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a706a761-3c78-405a-a324-a62c936f9216 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:09.846110 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6c0dd8-1abd-4f66-a5e4-a1d1931594ca None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:09.846492 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6c0dd8-1abd-4f66-a5e4-a1d1931594ca None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:09.846866 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6c0dd8-1abd-4f66-a5e4-a1d1931594ca None admin] PATH_INFO: `/v3/services` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:09.848240 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6c0dd8-1abd-4f66-a5e4-a1d1931594ca None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:07:09.848723 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6c0dd8-1abd-4f66-a5e4-a1d1931594ca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:09.850184 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6c0dd8-1abd-4f66-a5e4-a1d1931594ca None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:09.856633 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 385/768] 10.222.0.44 () {56 vars in 1124 bytes} [Wed Jun 29 11:07:09 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:07:09.862366 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 384/769] 10.222.0.44 () {56 vars in 1148 bytes} [Wed Jun 29 11:07:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:07:09.867684 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6c0dd8-1abd-4f66-a5e4-a1d1931594ca None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:09.869052 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a706a761-3c78-405a-a324-a62c936f9216 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:10.013702 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6c0dd8-1abd-4f66-a5e4-a1d1931594ca None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:10.053199 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5e7256-f9f7-4b62-9977-8929cd218672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:10.054247 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5e7256-f9f7-4b62-9977-8929cd218672 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:10.054646 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5e7256-f9f7-4b62-9977-8929cd218672 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:10.055023 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5e7256-f9f7-4b62-9977-8929cd218672 None admin] PATH_INFO: `/v3/services` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:10.055859 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5e7256-f9f7-4b62-9977-8929cd218672 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:07:10.056420 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5e7256-f9f7-4b62-9977-8929cd218672 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:10.058056 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5e7256-f9f7-4b62-9977-8929cd218672 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:10.060741 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a706a761-3c78-405a-a324-a62c936f9216 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:10.063879 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 386/770] 10.222.0.44 () {56 vars in 1124 bytes} [Wed Jun 29 11:07:09 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:07:10.065076 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ae1cc2-72ae-4778-9ed4-4fe430882965 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:10.070032 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ae1cc2-72ae-4778-9ed4-4fe430882965 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:10.070428 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ae1cc2-72ae-4778-9ed4-4fe430882965 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:10.076223 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 387/771] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:07:10 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:07:10.076223 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ae1cc2-72ae-4778-9ed4-4fe430882965 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:10.113246 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:10.114828 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727b31ee-b0fd-48d4-a5d7-1c24c38695d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:10.116080 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727b31ee-b0fd-48d4-a5d7-1c24c38695d8 None admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:10.116425 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727b31ee-b0fd-48d4-a5d7-1c24c38695d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:10.116807 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727b31ee-b0fd-48d4-a5d7-1c24c38695d8 None admin] PATH_INFO: `/v3/projects/ebe5dffa4bb8440286687daefb6c98f4/users/796aeeff16df4739bddec1f6dee61bfe/roles/e95c2ce63e114cc19a9dda6ad21e794f` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:10.118949 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727b31ee-b0fd-48d4-a5d7-1c24c38695d8 None admin] RBAC: Authorizing `identity:create_grant(project_id=ebe5dffa4bb8440286687daefb6c98f4, user_id=796aeeff16df4739bddec1f6dee61bfe, role_id=e95c2ce63e114cc19a9dda6ad21e794f)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:10.120522 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727b31ee-b0fd-48d4-a5d7-1c24c38695d8 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:10.192533 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ae1cc2-72ae-4778-9ed4-4fe430882965 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:10.200344 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 385/772] 10.222.0.44 () {58 vars in 1439 bytes} [Wed Jun 29 11:07:09 2022] PUT /identity/v3/projects/ebe5dffa4bb8440286687daefb6c98f4/users/796aeeff16df4739bddec1f6dee61bfe/roles/e95c2ce63e114cc19a9dda6ad21e794f => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:07:10.202984 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186ec08a-983d-4951-8276-8320daf742fe None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:10.203403 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186ec08a-983d-4951-8276-8320daf742fe None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:10.203796 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186ec08a-983d-4951-8276-8320daf742fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:10.238550 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-186ec08a-983d-4951-8276-8320daf742fe None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:07:10.240065 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-186ec08a-983d-4951-8276-8320daf742fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:07:10.260879 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f224c3fa-432e-4c2a-9104-a5f47326fee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:10.263675 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f224c3fa-432e-4c2a-9104-a5f47326fee9 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:10.264288 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f224c3fa-432e-4c2a-9104-a5f47326fee9 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:10.264776 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f224c3fa-432e-4c2a-9104-a5f47326fee9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:10.265801 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f224c3fa-432e-4c2a-9104-a5f47326fee9 None admin] RBAC: Adding query filter params (interface=public, service_id=d83daf50b8734b2891d7ec86a89cdd30, region_id=RegionOne) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:07:10.266300 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f224c3fa-432e-4c2a-9104-a5f47326fee9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:10.267941 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f224c3fa-432e-4c2a-9104-a5f47326fee9 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:10.287852 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 388/773] 10.222.0.44 () {56 vars in 1251 bytes} [Wed Jun 29 11:07:10 2022] GET /identity/v3/endpoints?service_id=d83daf50b8734b2891d7ec86a89cdd30&interface=public®ion_id=RegionOne => generated 194 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:07:10.288546 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:10.293086 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f224c3fa-432e-4c2a-9104-a5f47326fee9 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:10.338582 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 386/774] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:07:10 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 11:07:10.376457 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186ec08a-983d-4951-8276-8320daf742fe None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:10.430919 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:10.435129 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f224c3fa-432e-4c2a-9104-a5f47326fee9 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:10.477450 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae11c765-85ae-4930-833d-8712ab6122bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:10.478104 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae11c765-85ae-4930-833d-8712ab6122bd None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:10.478390 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae11c765-85ae-4930-833d-8712ab6122bd None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:10.478664 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae11c765-85ae-4930-833d-8712ab6122bd None admin] PATH_INFO: `/v3/services` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:10.479344 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae11c765-85ae-4930-833d-8712ab6122bd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:10.481441 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae11c765-85ae-4930-833d-8712ab6122bd None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:10.488407 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 389/775] 10.222.0.44 () {56 vars in 1087 bytes} [Wed Jun 29 11:07:10 2022] GET /identity/v3/services => generated 772 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:07:10.499220 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186ec08a-983d-4951-8276-8320daf742fe None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:10.541042 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80788006-2a35-4c83-921d-5134718199df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:10.541942 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80788006-2a35-4c83-921d-5134718199df None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:10.542335 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80788006-2a35-4c83-921d-5134718199df None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:10.542719 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80788006-2a35-4c83-921d-5134718199df None admin] PATH_INFO: `/v3/services/network` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:10.543524 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80788006-2a35-4c83-921d-5134718199df None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:10.545098 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80788006-2a35-4c83-921d-5134718199df None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:10.557776 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-80788006-2a35-4c83-921d-5134718199df None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 29 11:07:10.557776 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:07:10.557776 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 29 11:07:10.557776 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 29 11:07:10.557776 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 29 11:07:10.557776 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 29 11:07:10.557776 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 29 11:07:10.557776 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 11:07:10.557776 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 29 11:07:10.557776 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:07:10.557776 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 29 11:07:10.557776 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:07:10.557776 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:07:10.557776 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:07:10.557776 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:07:10.557776 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:07:10.557776 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:07:10.559207 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:07:10.559207 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:07:10.559207 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:07:10.559207 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:07:10.559207 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:07:10.559207 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:07:10.559207 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 29 11:07:10.559207 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 29 11:07:10.559207 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 29 11:07:10.559207 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 29 11:07:10.559207 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:07:10.559207 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:07:10.559207 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:07:10.559207 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:07:10.559207 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:07:10.559207 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:07:10.561369 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:07:10.561369 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jun 29 11:07:10.561369 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:07:10.561369 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:07:10.561369 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:07:10.561369 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:07:10.561369 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:07:10.561369 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:07:10.561369 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:07:10.561369 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:07:10.561369 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 29 11:07:10.561369 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 11:07:10.561369 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 29 11:07:10.561369 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:07:10.561369 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 387/776] 10.222.0.44 () {56 vars in 1111 bytes} [Wed Jun 29 11:07:10 2022] GET /identity/v3/services/network => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 11:07:10.569917 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae11c765-85ae-4930-833d-8712ab6122bd None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:10.698942 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae11c765-85ae-4930-833d-8712ab6122bd None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:10.733730 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7676e4d1-f371-42f1-bd4e-179822bead60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:10.734403 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7676e4d1-f371-42f1-bd4e-179822bead60 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:10.734684 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7676e4d1-f371-42f1-bd4e-179822bead60 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:10.734957 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7676e4d1-f371-42f1-bd4e-179822bead60 None admin] PATH_INFO: `/v3/services` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:10.735663 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7676e4d1-f371-42f1-bd4e-179822bead60 None admin] RBAC: Adding query filter params (name=network) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:07:10.735999 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7676e4d1-f371-42f1-bd4e-179822bead60 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:10.737365 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7676e4d1-f371-42f1-bd4e-179822bead60 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:10.743040 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 390/777] 10.222.0.44 () {56 vars in 1112 bytes} [Wed Jun 29 11:07:10 2022] GET /identity/v3/services?name=network => generated 124 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:07:10.748659 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80788006-2a35-4c83-921d-5134718199df None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:10.788290 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:10.873218 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80788006-2a35-4c83-921d-5134718199df None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:10.921436 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be67451-8a35-4cb7-8ff4-a17c1a4eeae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:10.922127 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be67451-8a35-4cb7-8ff4-a17c1a4eeae9 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:10.922413 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be67451-8a35-4cb7-8ff4-a17c1a4eeae9 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:10.922687 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be67451-8a35-4cb7-8ff4-a17c1a4eeae9 None admin] PATH_INFO: `/v3/services` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:10.923397 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be67451-8a35-4cb7-8ff4-a17c1a4eeae9 None admin] RBAC: Adding query filter params (type=network) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:07:10.923704 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be67451-8a35-4cb7-8ff4-a17c1a4eeae9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:10.925169 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be67451-8a35-4cb7-8ff4-a17c1a4eeae9 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:10.931716 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 388/778] 10.222.0.44 () {56 vars in 1112 bytes} [Wed Jun 29 11:07:10 2022] GET /identity/v3/services?type=network => generated 350 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:07:10.937631 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7676e4d1-f371-42f1-bd4e-179822bead60 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:10.983375 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:11.080187 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7676e4d1-f371-42f1-bd4e-179822bead60 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:11.130981 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5204fc0-ded3-4d61-af1e-d8d7bf63e74a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:11.131556 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5204fc0-ded3-4d61-af1e-d8d7bf63e74a None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:11.131851 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5204fc0-ded3-4d61-af1e-d8d7bf63e74a None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:11.132136 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5204fc0-ded3-4d61-af1e-d8d7bf63e74a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:11.132924 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5204fc0-ded3-4d61-af1e-d8d7bf63e74a None admin] RBAC: Adding query filter params (interface=public, service_id=c85a9d662d4148e2a3be924d0e4cc7e0, region_id=RegionOne) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:07:11.133231 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5204fc0-ded3-4d61-af1e-d8d7bf63e74a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:11.134588 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5204fc0-ded3-4d61-af1e-d8d7bf63e74a None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:11.140232 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 391/779] 10.222.0.44 () {56 vars in 1251 bytes} [Wed Jun 29 11:07:10 2022] GET /identity/v3/endpoints?service_id=c85a9d662d4148e2a3be924d0e4cc7e0&interface=public®ion_id=RegionOne => generated 194 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:07:11.145765 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be67451-8a35-4cb7-8ff4-a17c1a4eeae9 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:11.333501 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:11.347733 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be67451-8a35-4cb7-8ff4-a17c1a4eeae9 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:11.397450 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f28c732-9a41-4ade-ad6c-0cc75734b2be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:11.398047 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f28c732-9a41-4ade-ad6c-0cc75734b2be None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:11.398326 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f28c732-9a41-4ade-ad6c-0cc75734b2be None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:11.398598 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f28c732-9a41-4ade-ad6c-0cc75734b2be None admin] PATH_INFO: `/v3/services` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:11.399322 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f28c732-9a41-4ade-ad6c-0cc75734b2be None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:11.400692 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f28c732-9a41-4ade-ad6c-0cc75734b2be None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:11.411676 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 389/780] 10.222.0.44 () {56 vars in 1087 bytes} [Wed Jun 29 11:07:11 2022] GET /identity/v3/services => generated 772 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:07:11.478798 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182f0eca-f0a2-4643-8655-210220a5551c None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:11.479335 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182f0eca-f0a2-4643-8655-210220a5551c None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:11.479335 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182f0eca-f0a2-4643-8655-210220a5551c None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:11.480852 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 392/781] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:07:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:07:11.496716 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4d7da2-ebc5-4999-a87f-8965d3b3102d None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:11.497064 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4d7da2-ebc5-4999-a87f-8965d3b3102d None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:11.497269 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4d7da2-ebc5-4999-a87f-8965d3b3102d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:11.520849 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-6f4d7da2-ebc5-4999-a87f-8965d3b3102d None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:07:11.522062 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-6f4d7da2-ebc5-4999-a87f-8965d3b3102d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:07:11.618697 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 390/782] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:07:11 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 11:07:11.641913 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182f0eca-f0a2-4643-8655-210220a5551c None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:11.743605 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:11.767626 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182f0eca-f0a2-4643-8655-210220a5551c None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:11.822171 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9d950a-3ad4-426c-8ca3-cd977f00ac18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:11.822972 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9d950a-3ad4-426c-8ca3-cd977f00ac18 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:11.823254 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9d950a-3ad4-426c-8ca3-cd977f00ac18 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:11.823532 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9d950a-3ad4-426c-8ca3-cd977f00ac18 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:11.839860 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9d950a-3ad4-426c-8ca3-cd977f00ac18 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 11:07:11.848149 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-dd9d950a-3ad4-426c-8ca3-cd977f00ac18 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 29 11:07:11.848149 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:07:11.848149 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:07:11.848149 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:07:11.848149 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:07:11.848149 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:07:11.848149 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:07:11.848149 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:07:11.848149 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:07:11.848149 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:07:11.848149 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:07:11.848149 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:07:11.848149 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 11:07:11.848149 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 11:07:11.848149 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 11:07:11.848149 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 11:07:11.849083 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 11:07:11.849083 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 11:07:11.849083 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 11:07:11.849083 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 11:07:11.849083 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:07:11.849083 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:07:11.849083 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:07:11.849083 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:07:11.849083 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:07:11.849083 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:07:11.849083 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:07:11.849083 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jun 29 11:07:11.849083 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:07:11.849083 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:07:11.849083 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:07:11.849083 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:07:11.850318 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:07:11.850318 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:07:11.850318 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:07:11.850318 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:07:11.850318 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 11:07:11.850318 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 11:07:11.850318 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 11:07:11.850318 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 11:07:11.850318 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 11:07:11.850318 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 11:07:11.850318 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 29 11:07:11.850318 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jun 29 11:07:11.850318 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 393/783] 10.222.0.44 () {56 vars in 1102 bytes} [Wed Jun 29 11:07:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:07:11.856919 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4d7da2-ebc5-4999-a87f-8965d3b3102d None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:11.977849 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:12.000601 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4d7da2-ebc5-4999-a87f-8965d3b3102d None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:12.052519 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdac713-d525-4262-b1c4-27b3f8c0a914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:12.053202 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdac713-d525-4262-b1c4-27b3f8c0a914 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:12.053491 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdac713-d525-4262-b1c4-27b3f8c0a914 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:12.053769 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdac713-d525-4262-b1c4-27b3f8c0a914 None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:12.054515 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdac713-d525-4262-b1c4-27b3f8c0a914 None admin] RBAC: Adding query filter params (name=service) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:07:12.054823 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdac713-d525-4262-b1c4-27b3f8c0a914 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:12.056220 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdac713-d525-4262-b1c4-27b3f8c0a914 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:12.065431 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 391/784] 10.222.0.44 () {56 vars in 1103 bytes} [Wed Jun 29 11:07:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:07:12.071137 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9d950a-3ad4-426c-8ca3-cd977f00ac18 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:12.234947 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9d950a-3ad4-426c-8ca3-cd977f00ac18 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:12.286360 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a057d4c0-8247-42ff-a988-692be10bf24a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:12.287085 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a057d4c0-8247-42ff-a988-692be10bf24a None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:12.287379 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a057d4c0-8247-42ff-a988-692be10bf24a None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:12.287660 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a057d4c0-8247-42ff-a988-692be10bf24a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:12.304069 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a057d4c0-8247-42ff-a988-692be10bf24a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:12.305526 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a057d4c0-8247-42ff-a988-692be10bf24a None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:12.312202 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:12.326991 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-a057d4c0-8247-42ff-a988-692be10bf24a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 11:07:12.326991 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:07:12.326991 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:07:12.326991 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:07:12.326991 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:07:12.326991 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:07:12.326991 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:07:12.326991 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:07:12.326991 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:07:12.326991 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:07:12.326991 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:07:12.326991 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:07:12.326991 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 11:07:12.326991 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 11:07:12.326991 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 11:07:12.328462 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 11:07:12.328462 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:07:12.328462 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:07:12.328462 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:07:12.328462 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:07:12.328462 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:07:12.328462 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:07:12.328462 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:07:12.328462 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jun 29 11:07:12.328462 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:07:12.328462 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:07:12.328462 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:07:12.328462 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:07:12.328462 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:07:12.328462 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:07:12.328462 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:07:12.328462 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:07:12.329365 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 11:07:12.329365 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 11:07:12.329365 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 11:07:12.329365 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jun 29 11:07:12.329365 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 394/785] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jun 29 11:07:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:07:12.336567 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdac713-d525-4262-b1c4-27b3f8c0a914 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:12.349994 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f2c480-08c9-4a1a-bde5-4d2b5b198500 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:12.350583 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f2c480-08c9-4a1a-bde5-4d2b5b198500 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:12.351030 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f2c480-08c9-4a1a-bde5-4d2b5b198500 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:12.353001 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 395/786] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:07:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:07:12.358174 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8aa4b7-2959-4b1a-a3cb-c27b09fa0165 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:12.358692 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8aa4b7-2959-4b1a-a3cb-c27b09fa0165 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:12.359161 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8aa4b7-2959-4b1a-a3cb-c27b09fa0165 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:12.389179 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-2e8aa4b7-2959-4b1a-a3cb-c27b09fa0165 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:07:12.393768 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-2e8aa4b7-2959-4b1a-a3cb-c27b09fa0165 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:07:12.523139 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdac713-d525-4262-b1c4-27b3f8c0a914 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:12.550080 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 396/787] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:07:12 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 29 11:07:12.557857 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900dc3e4-637b-41ec-bca0-38baae9489ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:12.558517 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900dc3e4-637b-41ec-bca0-38baae9489ee None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:12.558798 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900dc3e4-637b-41ec-bca0-38baae9489ee None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:12.559070 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900dc3e4-637b-41ec-bca0-38baae9489ee None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:12.559783 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900dc3e4-637b-41ec-bca0-38baae9489ee None admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:07:12.560116 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900dc3e4-637b-41ec-bca0-38baae9489ee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:12.561473 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900dc3e4-637b-41ec-bca0-38baae9489ee None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:12.575128 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 392/788] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Jun 29 11:07:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:07:12.580774 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8aa4b7-2959-4b1a-a3cb-c27b09fa0165 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:12.591123 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900dc3e4-637b-41ec-bca0-38baae9489ee None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:12.595038 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:12.708935 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8aa4b7-2959-4b1a-a3cb-c27b09fa0165 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:12.776249 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:12.787425 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25e3942-9174-4577-8866-4e152ecbdde8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:12.788283 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25e3942-9174-4577-8866-4e152ecbdde8 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:12.788571 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25e3942-9174-4577-8866-4e152ecbdde8 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:12.788849 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25e3942-9174-4577-8866-4e152ecbdde8 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:12.797098 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25e3942-9174-4577-8866-4e152ecbdde8 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:12.798656 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25e3942-9174-4577-8866-4e152ecbdde8 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:12.815057 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-c25e3942-9174-4577-8866-4e152ecbdde8 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 29 11:07:12.815057 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:07:12.815057 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:07:12.815057 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:07:12.815057 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:07:12.815057 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:07:12.815057 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:07:12.815057 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:07:12.815057 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:07:12.815057 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:07:12.815057 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:07:12.815057 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:07:12.815057 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 11:07:12.815057 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 11:07:12.815057 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 11:07:12.815057 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 11:07:12.816029 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:07:12.816029 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:07:12.816029 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 11:07:12.816029 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 11:07:12.816029 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 11:07:12.816029 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 11:07:12.816029 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:07:12.816029 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:07:12.816029 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:07:12.816029 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:07:12.816029 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:07:12.816029 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jun 29 11:07:12.816029 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:07:12.816029 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:07:12.816029 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:07:12.816029 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:07:12.816029 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:07:12.816029 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:07:12.817521 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:07:12.817521 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:07:12.817521 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 11:07:12.817521 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 11:07:12.817521 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 11:07:12.817521 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 11:07:12.817521 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 11:07:12.817521 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 11:07:12.817521 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 29 11:07:12.817521 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jun 29 11:07:12.817521 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 397/789] 10.222.0.44 () {56 vars in 1117 bytes} [Wed Jun 29 11:07:12 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:07:12.818341 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138e73f3-61fc-4a86-86c6-fc0f9777dcce None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:12.818625 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138e73f3-61fc-4a86-86c6-fc0f9777dcce None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:12.818905 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138e73f3-61fc-4a86-86c6-fc0f9777dcce None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:12.823951 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 398/790] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:07:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:07:12.824716 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138e73f3-61fc-4a86-86c6-fc0f9777dcce None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:12.866373 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900dc3e4-637b-41ec-bca0-38baae9489ee None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:12.904541 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c1c4e9-963b-4ba6-8aaf-6472f6ed94b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:12.905341 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c1c4e9-963b-4ba6-8aaf-6472f6ed94b6 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:12.905646 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c1c4e9-963b-4ba6-8aaf-6472f6ed94b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:12.905896 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c1c4e9-963b-4ba6-8aaf-6472f6ed94b6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:12.925319 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c1c4e9-963b-4ba6-8aaf-6472f6ed94b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 11:07:12.933091 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-b8c1c4e9-963b-4ba6-8aaf-6472f6ed94b6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 11:07:12.933091 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:07:12.933091 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:07:12.933091 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:07:12.933091 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:07:12.933091 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:07:12.933091 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:07:12.933091 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:07:12.933091 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:07:12.933091 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:07:12.933091 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:07:12.933091 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:07:12.933091 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 11:07:12.933091 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 11:07:12.933091 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 11:07:12.933091 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 11:07:12.934124 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 11:07:12.934124 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 11:07:12.934124 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 11:07:12.934124 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 11:07:12.934124 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:07:12.934124 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:07:12.934124 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:07:12.934124 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:07:12.934124 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:07:12.934124 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:07:12.934124 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:07:12.934124 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jun 29 11:07:12.934124 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:07:12.934124 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:07:12.934124 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:07:12.934124 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:07:12.934969 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:07:12.934969 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:07:12.934969 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:07:12.934969 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:07:12.934969 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 11:07:12.934969 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 11:07:12.934969 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 11:07:12.934969 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 11:07:12.934969 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 11:07:12.934969 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 11:07:12.934969 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 11:07:12.934969 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:07:12.934969 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 393/791] 10.222.0.44 () {56 vars in 1096 bytes} [Wed Jun 29 11:07:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 344 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 11:07:12.936183 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4541e9da-f1e8-423e-b3a4-986687e9d34e None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:12.936183 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4541e9da-f1e8-423e-b3a4-986687e9d34e None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:12.936356 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4541e9da-f1e8-423e-b3a4-986687e9d34e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:12.954869 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-4541e9da-f1e8-423e-b3a4-986687e9d34e None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:07:12.956129 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-4541e9da-f1e8-423e-b3a4-986687e9d34e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:07:12.962599 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138e73f3-61fc-4a86-86c6-fc0f9777dcce None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:12.998561 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33661098-1b80-4cd5-93ca-4748b6a98e5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:12.999583 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33661098-1b80-4cd5-93ca-4748b6a98e5e None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:13.000180 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33661098-1b80-4cd5-93ca-4748b6a98e5e None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:13.000658 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33661098-1b80-4cd5-93ca-4748b6a98e5e None admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:13.001754 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33661098-1b80-4cd5-93ca-4748b6a98e5e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:07:13.002311 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33661098-1b80-4cd5-93ca-4748b6a98e5e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:13.006993 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33661098-1b80-4cd5-93ca-4748b6a98e5e None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:13.019453 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:13.026294 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 399/792] 10.222.0.44 () {56 vars in 1149 bytes} [Wed Jun 29 11:07:12 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:07:13.027481 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33661098-1b80-4cd5-93ca-4748b6a98e5e None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:13.028906 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 394/793] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:07:12 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 11:07:13.032057 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4541e9da-f1e8-423e-b3a4-986687e9d34e None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:13.100432 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:13.155993 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33661098-1b80-4cd5-93ca-4748b6a98e5e None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:13.156694 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4541e9da-f1e8-423e-b3a4-986687e9d34e None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:13.192557 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52106291-755e-4593-8a6c-2cd13e277445 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:13.193462 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52106291-755e-4593-8a6c-2cd13e277445 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:13.193897 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52106291-755e-4593-8a6c-2cd13e277445 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:13.194351 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52106291-755e-4593-8a6c-2cd13e277445 None admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:13.195237 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52106291-755e-4593-8a6c-2cd13e277445 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:07:13.195691 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52106291-755e-4593-8a6c-2cd13e277445 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:13.197494 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52106291-755e-4593-8a6c-2cd13e277445 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:13.203578 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9d07a7-18c3-4d83-a900-7cb4ef6e200c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:13.204509 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9d07a7-18c3-4d83-a900-7cb4ef6e200c None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:13.204884 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9d07a7-18c3-4d83-a900-7cb4ef6e200c None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:13.205253 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9d07a7-18c3-4d83-a900-7cb4ef6e200c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:13.207467 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 400/794] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Jun 29 11:07:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:07:13.208522 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52106291-755e-4593-8a6c-2cd13e277445 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:13.216720 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9d07a7-18c3-4d83-a900-7cb4ef6e200c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:13.218284 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9d07a7-18c3-4d83-a900-7cb4ef6e200c None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:13.236553 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-ac9d07a7-18c3-4d83-a900-7cb4ef6e200c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 11:07:13.236553 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:07:13.236553 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:07:13.236553 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:07:13.236553 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:07:13.236553 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:07:13.236553 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:07:13.236553 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:07:13.236553 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:07:13.236553 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:07:13.236553 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:07:13.236553 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:07:13.236553 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 11:07:13.236553 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 11:07:13.236553 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 11:07:13.237656 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 11:07:13.237656 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:07:13.237656 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:07:13.237656 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:07:13.237656 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:07:13.237656 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:07:13.237656 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:07:13.237656 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:07:13.237656 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jun 29 11:07:13.237656 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:07:13.237656 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:07:13.237656 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:07:13.237656 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:07:13.237656 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:07:13.237656 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:07:13.237656 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:07:13.237656 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:07:13.239331 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 11:07:13.239331 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 11:07:13.239331 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 11:07:13.239331 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:07:13.241018 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 395/795] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jun 29 11:07:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:07:13.241872 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9d07a7-18c3-4d83-a900-7cb4ef6e200c None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:13.289610 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:13.318188 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52106291-755e-4593-8a6c-2cd13e277445 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:13.327510 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:13.346312 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9d07a7-18c3-4d83-a900-7cb4ef6e200c None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:13.354132 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c969c66-d20e-4aea-8afc-664c4a794000 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:13.355251 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c969c66-d20e-4aea-8afc-664c4a794000 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:13.355699 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c969c66-d20e-4aea-8afc-664c4a794000 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:13.356244 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c969c66-d20e-4aea-8afc-664c4a794000 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:13.370670 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c969c66-d20e-4aea-8afc-664c4a794000 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 11:07:13.378799 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-6c969c66-d20e-4aea-8afc-664c4a794000 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 11:07:13.378799 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:07:13.378799 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:07:13.378799 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:07:13.378799 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:07:13.378799 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:07:13.378799 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:07:13.378799 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:07:13.378799 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:07:13.378799 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:07:13.378799 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:07:13.378799 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:07:13.378799 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 11:07:13.378799 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 11:07:13.378799 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 11:07:13.378799 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 11:07:13.379913 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 11:07:13.379913 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 11:07:13.379913 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 11:07:13.379913 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 11:07:13.379913 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:07:13.379913 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:07:13.379913 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:07:13.379913 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:07:13.379913 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:07:13.379913 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:07:13.379913 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:07:13.379913 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jun 29 11:07:13.379913 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:07:13.379913 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:07:13.379913 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:07:13.379913 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:07:13.380835 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:07:13.380835 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:07:13.380835 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:07:13.380835 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:07:13.380835 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 11:07:13.380835 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 11:07:13.380835 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 11:07:13.380835 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 11:07:13.380835 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 11:07:13.380835 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 11:07:13.380835 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 11:07:13.380835 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jun 29 11:07:13.383135 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 401/796] 10.222.0.44 () {56 vars in 1096 bytes} [Wed Jun 29 11:07:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:07:13.384091 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c969c66-d20e-4aea-8afc-664c4a794000 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:13.388394 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9823622e-8b88-4433-9973-c291c755d454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:13.389201 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9823622e-8b88-4433-9973-c291c755d454 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:13.389485 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9823622e-8b88-4433-9973-c291c755d454 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:13.389757 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9823622e-8b88-4433-9973-c291c755d454 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:13.403423 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9823622e-8b88-4433-9973-c291c755d454 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:13.404756 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9823622e-8b88-4433-9973-c291c755d454 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:13.424509 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-9823622e-8b88-4433-9973-c291c755d454 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 11:07:13.424509 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:07:13.424509 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:07:13.424509 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:07:13.424509 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:07:13.424509 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:07:13.424509 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:07:13.424509 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:07:13.424509 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:07:13.424509 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:07:13.424509 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:07:13.424509 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:07:13.424509 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 11:07:13.424509 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 11:07:13.424509 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 11:07:13.425524 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 11:07:13.425524 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:07:13.425524 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:07:13.425524 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:07:13.425524 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:07:13.425524 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:07:13.425524 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:07:13.425524 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:07:13.425524 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jun 29 11:07:13.425524 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:07:13.425524 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:07:13.425524 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:07:13.425524 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:07:13.425524 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:07:13.425524 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:07:13.425524 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:07:13.425524 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:07:13.427041 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 11:07:13.427041 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 11:07:13.427041 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 11:07:13.427041 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:07:13.427041 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 396/797] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jun 29 11:07:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:07:13.427324 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604d7d04-9bad-482c-9354-0313fc023637 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:13.427495 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604d7d04-9bad-482c-9354-0313fc023637 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:13.427723 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604d7d04-9bad-482c-9354-0313fc023637 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:13.432946 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 397/798] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:07:13 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:07:13.433591 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604d7d04-9bad-482c-9354-0313fc023637 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:13.496703 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c969c66-d20e-4aea-8afc-664c4a794000 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:13.534456 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0630d964-ddb7-4886-baab-dc28fbb3409d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:13.535143 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0630d964-ddb7-4886-baab-dc28fbb3409d None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:13.535428 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0630d964-ddb7-4886-baab-dc28fbb3409d None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:13.535706 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0630d964-ddb7-4886-baab-dc28fbb3409d None admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:13.536453 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0630d964-ddb7-4886-baab-dc28fbb3409d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:07:13.536761 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0630d964-ddb7-4886-baab-dc28fbb3409d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:13.538212 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0630d964-ddb7-4886-baab-dc28fbb3409d None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:13.539488 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:13.544206 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604d7d04-9bad-482c-9354-0313fc023637 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:13.554258 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 402/799] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Jun 29 11:07:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:07:13.554258 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0630d964-ddb7-4886-baab-dc28fbb3409d None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:13.584288 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:13.584991 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9781e61-f5a3-4c4d-a936-1324b6aeaa89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:13.585663 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9781e61-f5a3-4c4d-a936-1324b6aeaa89 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:13.585947 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9781e61-f5a3-4c4d-a936-1324b6aeaa89 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:13.586225 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9781e61-f5a3-4c4d-a936-1324b6aeaa89 None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:13.586974 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9781e61-f5a3-4c4d-a936-1324b6aeaa89 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:07:13.587282 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9781e61-f5a3-4c4d-a936-1324b6aeaa89 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:13.588667 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9781e61-f5a3-4c4d-a936-1324b6aeaa89 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:13.601807 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 398/800] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Jun 29 11:07:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:07:13.601807 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c569d74d-4ba4-45b3-be56-443e6b4af252 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:13.601807 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c569d74d-4ba4-45b3-be56-443e6b4af252 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:13.601807 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c569d74d-4ba4-45b3-be56-443e6b4af252 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:13.616008 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-c569d74d-4ba4-45b3-be56-443e6b4af252 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:07:13.617135 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-c569d74d-4ba4-45b3-be56-443e6b4af252 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:07:13.663802 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0630d964-ddb7-4886-baab-dc28fbb3409d None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:13.695870 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 399/801] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:07:13 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 11:07:13.695870 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c569d74d-4ba4-45b3-be56-443e6b4af252 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:13.709050 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072e1ebc-f8e4-4493-8ad9-57bd2902c8ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:13.709797 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072e1ebc-f8e4-4493-8ad9-57bd2902c8ef None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:13.710170 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072e1ebc-f8e4-4493-8ad9-57bd2902c8ef None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:13.710622 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072e1ebc-f8e4-4493-8ad9-57bd2902c8ef None admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:13.711517 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072e1ebc-f8e4-4493-8ad9-57bd2902c8ef None admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:07:13.711851 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072e1ebc-f8e4-4493-8ad9-57bd2902c8ef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:13.713273 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072e1ebc-f8e4-4493-8ad9-57bd2902c8ef None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:13.730070 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 403/802] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Jun 29 11:07:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:07:13.730786 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072e1ebc-f8e4-4493-8ad9-57bd2902c8ef None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:13.788901 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:13.798745 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c569d74d-4ba4-45b3-be56-443e6b4af252 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:13.825954 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:13.835037 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f8458a-11d8-48ff-8a45-1b09377b597b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:13.836060 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f8458a-11d8-48ff-8a45-1b09377b597b None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:13.836521 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f8458a-11d8-48ff-8a45-1b09377b597b None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:13.837075 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f8458a-11d8-48ff-8a45-1b09377b597b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:13.837572 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072e1ebc-f8e4-4493-8ad9-57bd2902c8ef None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:13.844768 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f8458a-11d8-48ff-8a45-1b09377b597b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:13.846572 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f8458a-11d8-48ff-8a45-1b09377b597b None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:13.860113 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-93f8458a-11d8-48ff-8a45-1b09377b597b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 11:07:13.860113 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:07:13.860113 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:07:13.860113 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:07:13.860113 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:07:13.860113 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:07:13.860113 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:07:13.860113 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:07:13.860113 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:07:13.860113 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:07:13.860113 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:07:13.860113 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:07:13.860113 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 11:07:13.860113 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 11:07:13.860113 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 11:07:13.861169 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 11:07:13.861169 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:07:13.861169 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:07:13.861169 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:07:13.861169 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:07:13.861169 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:07:13.861169 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:07:13.861169 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:07:13.861169 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jun 29 11:07:13.861169 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:07:13.861169 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:07:13.861169 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:07:13.861169 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:07:13.861169 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:07:13.861169 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:07:13.861169 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:07:13.861169 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:07:13.862269 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 11:07:13.862269 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 11:07:13.862269 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 11:07:13.862269 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 11:07:13.862269 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 11:07:13.862269 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 11:07:13.862269 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 11:07:13.862269 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:07:13.863614 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 400/803] 10.222.0.44 () {56 vars in 1111 bytes} [Wed Jun 29 11:07:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:07:13.863614 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f8458a-11d8-48ff-8a45-1b09377b597b None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:13.879711 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66aee49-8bf6-4d79-a8c1-68daee0ee03f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:13.880477 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66aee49-8bf6-4d79-a8c1-68daee0ee03f None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:13.880758 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66aee49-8bf6-4d79-a8c1-68daee0ee03f None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:13.881032 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66aee49-8bf6-4d79-a8c1-68daee0ee03f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:13.892022 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66aee49-8bf6-4d79-a8c1-68daee0ee03f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:13.893519 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66aee49-8bf6-4d79-a8c1-68daee0ee03f None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:13.912497 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-c66aee49-8bf6-4d79-a8c1-68daee0ee03f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 11:07:13.912497 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:07:13.912497 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:07:13.912497 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:07:13.912497 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:07:13.912497 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:07:13.912497 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:07:13.912497 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:07:13.912497 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:07:13.912497 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:07:13.912497 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:07:13.912497 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:07:13.912497 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 11:07:13.912497 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 11:07:13.912497 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 11:07:13.913467 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 11:07:13.913467 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:07:13.913467 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:07:13.913467 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:07:13.913467 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:07:13.913467 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:07:13.913467 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:07:13.913467 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:07:13.913467 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jun 29 11:07:13.913467 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:07:13.913467 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:07:13.913467 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:07:13.913467 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:07:13.913467 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:07:13.913467 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:07:13.913467 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:07:13.913467 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:07:13.919115 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 11:07:13.919115 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 11:07:13.919115 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 11:07:13.919115 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jun 29 11:07:13.919115 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 404/804] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jun 29 11:07:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:07:13.919115 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66aee49-8bf6-4d79-a8c1-68daee0ee03f None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:13.972645 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f8458a-11d8-48ff-8a45-1b09377b597b None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:14.006838 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb47e5e3-6d4f-4a3b-a9ef-f8b009c07303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:14.007640 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb47e5e3-6d4f-4a3b-a9ef-f8b009c07303 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:14.007964 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb47e5e3-6d4f-4a3b-a9ef-f8b009c07303 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:14.008241 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb47e5e3-6d4f-4a3b-a9ef-f8b009c07303 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:14.008884 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb47e5e3-6d4f-4a3b-a9ef-f8b009c07303 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:14.010254 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb47e5e3-6d4f-4a3b-a9ef-f8b009c07303 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:14.017873 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66aee49-8bf6-4d79-a8c1-68daee0ee03f None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:14.022056 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-fb47e5e3-6d4f-4a3b-a9ef-f8b009c07303 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 29 11:07:14.022056 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:07:14.022056 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 29 11:07:14.022056 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 29 11:07:14.022056 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 29 11:07:14.022056 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 29 11:07:14.022056 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 29 11:07:14.022056 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 11:07:14.022056 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 29 11:07:14.022056 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:07:14.022056 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 29 11:07:14.022056 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:07:14.022056 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:07:14.022056 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:07:14.022056 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:07:14.022056 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:07:14.022968 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:07:14.022968 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:07:14.022968 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:07:14.022968 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:07:14.022968 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:07:14.022968 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:07:14.022968 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:07:14.022968 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 29 11:07:14.022968 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 29 11:07:14.022968 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 29 11:07:14.022968 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 29 11:07:14.022968 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:07:14.022968 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:07:14.022968 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:07:14.022968 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:07:14.022968 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:07:14.023809 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:07:14.023809 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:07:14.023809 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jun 29 11:07:14.023809 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:07:14.023809 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:07:14.023809 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:07:14.023809 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:07:14.023809 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:07:14.023809 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:07:14.023809 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:07:14.023809 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:07:14.023809 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 29 11:07:14.023809 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 11:07:14.023809 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 29 11:07:14.023809 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:07:14.023809 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 401/805] 10.222.0.44 () {56 vars in 1129 bytes} [Wed Jun 29 11:07:13 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:07:14.024990 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb47e5e3-6d4f-4a3b-a9ef-f8b009c07303 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:14.039956 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:14.054643 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94a06de-038c-460a-84af-6a977825ab68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:14.055434 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94a06de-038c-460a-84af-6a977825ab68 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:14.055712 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94a06de-038c-460a-84af-6a977825ab68 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:14.056008 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94a06de-038c-460a-84af-6a977825ab68 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:14.063268 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94a06de-038c-460a-84af-6a977825ab68 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:14.064803 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94a06de-038c-460a-84af-6a977825ab68 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:14.067627 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:14.079814 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-d94a06de-038c-460a-84af-6a977825ab68 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 29 11:07:14.079814 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:07:14.079814 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:07:14.079814 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:07:14.079814 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:07:14.079814 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:07:14.079814 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:07:14.079814 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:07:14.079814 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:07:14.079814 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:07:14.079814 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:07:14.079814 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:07:14.079814 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 11:07:14.079814 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 11:07:14.079814 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 11:07:14.079814 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 11:07:14.086463 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:07:14.086463 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:07:14.086463 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 11:07:14.086463 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 11:07:14.086463 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 11:07:14.086463 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 11:07:14.086463 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:07:14.086463 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:07:14.086463 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:07:14.086463 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:07:14.086463 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:07:14.086463 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jun 29 11:07:14.086463 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:07:14.086463 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:07:14.086463 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:07:14.086463 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:07:14.086463 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:07:14.086463 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:07:14.087406 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:07:14.087406 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:07:14.087406 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 11:07:14.087406 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 11:07:14.087406 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 11:07:14.087406 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 11:07:14.087406 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 11:07:14.087406 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 11:07:14.087406 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 29 11:07:14.087406 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jun 29 11:07:14.087406 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 405/806] 10.222.0.44 () {56 vars in 1096 bytes} [Wed Jun 29 11:07:13 2022] GET /identity/v3/users/swift => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:07:14.087406 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60563b5d-75ae-4bb4-b53c-898eafe9c8b7 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:14.087406 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60563b5d-75ae-4bb4-b53c-898eafe9c8b7 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:14.088160 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60563b5d-75ae-4bb4-b53c-898eafe9c8b7 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:14.088160 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 406/807] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:07:14 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:07:14.088160 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60563b5d-75ae-4bb4-b53c-898eafe9c8b7 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:14.126787 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb47e5e3-6d4f-4a3b-a9ef-f8b009c07303 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:14.161983 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e771399-bcbb-4aae-88c4-ab3b8b0d3b31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:14.162564 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e771399-bcbb-4aae-88c4-ab3b8b0d3b31 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:14.162849 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e771399-bcbb-4aae-88c4-ab3b8b0d3b31 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:14.163133 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e771399-bcbb-4aae-88c4-ab3b8b0d3b31 None admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:14.163942 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e771399-bcbb-4aae-88c4-ab3b8b0d3b31 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:07:14.164251 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e771399-bcbb-4aae-88c4-ab3b8b0d3b31 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:14.165677 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e771399-bcbb-4aae-88c4-ab3b8b0d3b31 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:14.177954 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 402/808] 10.222.0.44 () {56 vars in 1148 bytes} [Wed Jun 29 11:07:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:07:14.178770 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e771399-bcbb-4aae-88c4-ab3b8b0d3b31 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:14.194421 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60563b5d-75ae-4bb4-b53c-898eafe9c8b7 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:14.227615 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4163b664-d55c-48a3-a581-fb023a429754 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:14.228303 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4163b664-d55c-48a3-a581-fb023a429754 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:14.228583 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4163b664-d55c-48a3-a581-fb023a429754 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:14.228856 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4163b664-d55c-48a3-a581-fb023a429754 None admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:14.229570 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4163b664-d55c-48a3-a581-fb023a429754 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:07:14.229872 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4163b664-d55c-48a3-a581-fb023a429754 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:14.231229 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4163b664-d55c-48a3-a581-fb023a429754 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:14.242543 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 407/809] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Jun 29 11:07:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:07:14.243704 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7120cde8-be19-4743-96fb-cd706da4f4f0 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:14.244021 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7120cde8-be19-4743-96fb-cd706da4f4f0 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:14.244297 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7120cde8-be19-4743-96fb-cd706da4f4f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:14.256504 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:14.260276 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-7120cde8-be19-4743-96fb-cd706da4f4f0 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:07:14.261460 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-7120cde8-be19-4743-96fb-cd706da4f4f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:07:14.280779 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e771399-bcbb-4aae-88c4-ab3b8b0d3b31 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:14.313939 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea9ac49-96d9-46ae-b2b9-b28eb3a10b13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:14.314597 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea9ac49-96d9-46ae-b2b9-b28eb3a10b13 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:14.314876 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea9ac49-96d9-46ae-b2b9-b28eb3a10b13 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:14.315153 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea9ac49-96d9-46ae-b2b9-b28eb3a10b13 None admin] PATH_INFO: `/v3/services` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:14.315612 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:14.315906 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea9ac49-96d9-46ae-b2b9-b28eb3a10b13 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:07:14.316686 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea9ac49-96d9-46ae-b2b9-b28eb3a10b13 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:14.318037 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea9ac49-96d9-46ae-b2b9-b28eb3a10b13 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:14.322812 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 403/810] 10.222.0.44 () {56 vars in 1124 bytes} [Wed Jun 29 11:07:14 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:07:14.323575 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea9ac49-96d9-46ae-b2b9-b28eb3a10b13 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:14.328601 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 408/811] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:07:14 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 11:07:14.329390 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7120cde8-be19-4743-96fb-cd706da4f4f0 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:14.425428 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea9ac49-96d9-46ae-b2b9-b28eb3a10b13 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:14.427494 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7120cde8-be19-4743-96fb-cd706da4f4f0 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:14.458488 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506990f2-b8ab-4d97-a61d-0cba00e7011b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:14.459155 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506990f2-b8ab-4d97-a61d-0cba00e7011b None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:14.459435 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506990f2-b8ab-4d97-a61d-0cba00e7011b None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:14.459713 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506990f2-b8ab-4d97-a61d-0cba00e7011b None admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:14.460574 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506990f2-b8ab-4d97-a61d-0cba00e7011b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:07:14.460676 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ddbc2a-24d6-4766-850a-3167a896dc21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:14.460882 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506990f2-b8ab-4d97-a61d-0cba00e7011b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:14.461468 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ddbc2a-24d6-4766-850a-3167a896dc21 None admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:14.461748 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ddbc2a-24d6-4766-850a-3167a896dc21 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:14.462020 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ddbc2a-24d6-4766-850a-3167a896dc21 None admin] PATH_INFO: `/v3/projects/ebe5dffa4bb8440286687daefb6c98f4/users/55487c2107a44c15b656b4865ee0d4fa/roles/a8212be3426741cbb1b38b1646c66405` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:14.462288 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506990f2-b8ab-4d97-a61d-0cba00e7011b None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:14.470283 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:14.475724 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ddbc2a-24d6-4766-850a-3167a896dc21 None admin] RBAC: Authorizing `identity:create_grant(project_id=ebe5dffa4bb8440286687daefb6c98f4, user_id=55487c2107a44c15b656b4865ee0d4fa, role_id=a8212be3426741cbb1b38b1646c66405)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:14.481782 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ddbc2a-24d6-4766-850a-3167a896dc21 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:14.481893 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 404/812] 10.222.0.44 () {56 vars in 1135 bytes} [Wed Jun 29 11:07:14 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:07:14.481893 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506990f2-b8ab-4d97-a61d-0cba00e7011b None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:14.574538 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506990f2-b8ab-4d97-a61d-0cba00e7011b None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:14.584958 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 409/813] 10.222.0.44 () {58 vars in 1439 bytes} [Wed Jun 29 11:07:14 2022] PUT /identity/v3/projects/ebe5dffa4bb8440286687daefb6c98f4/users/55487c2107a44c15b656b4865ee0d4fa/roles/a8212be3426741cbb1b38b1646c66405 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:07:14.585846 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ddbc2a-24d6-4766-850a-3167a896dc21 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:14.611318 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c6050e-85a5-47cf-a2ae-305a174cfff2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:14.612154 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c6050e-85a5-47cf-a2ae-305a174cfff2 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:14.612457 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c6050e-85a5-47cf-a2ae-305a174cfff2 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:14.613654 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c6050e-85a5-47cf-a2ae-305a174cfff2 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:14.621543 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c6050e-85a5-47cf-a2ae-305a174cfff2 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:14.623004 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c6050e-85a5-47cf-a2ae-305a174cfff2 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:14.633209 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:14.639096 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-98c6050e-85a5-47cf-a2ae-305a174cfff2 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 29 11:07:14.639096 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:07:14.639096 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:07:14.639096 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:07:14.639096 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:07:14.639096 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:07:14.639096 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:07:14.639096 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:07:14.639096 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:07:14.639096 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:07:14.639096 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:07:14.639096 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:07:14.639096 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 11:07:14.639096 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 11:07:14.639096 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 11:07:14.639096 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 11:07:14.640179 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:07:14.640179 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:07:14.640179 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 11:07:14.640179 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 11:07:14.640179 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 11:07:14.640179 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 11:07:14.640179 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:07:14.640179 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:07:14.640179 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:07:14.640179 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:07:14.640179 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:07:14.640179 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jun 29 11:07:14.640179 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:07:14.640179 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:07:14.640179 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:07:14.640179 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:07:14.640179 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:07:14.640179 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:07:14.641681 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:07:14.641681 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:07:14.641681 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 11:07:14.641681 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 11:07:14.641681 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 11:07:14.641681 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 11:07:14.641681 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 11:07:14.641681 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 11:07:14.641681 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 29 11:07:14.641681 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:07:14.641681 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 405/814] 10.222.0.44 () {56 vars in 1093 bytes} [Wed Jun 29 11:07:14 2022] GET /identity/v3/users/nova => generated 82 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:07:14.641681 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c6050e-85a5-47cf-a2ae-305a174cfff2 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:14.704983 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ddbc2a-24d6-4766-850a-3167a896dc21 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:14.712508 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:14.750021 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a1d046-ddb6-4c1b-906c-cb234d52c90e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:14.750696 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a1d046-ddb6-4c1b-906c-cb234d52c90e None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:14.750985 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a1d046-ddb6-4c1b-906c-cb234d52c90e None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:14.751258 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a1d046-ddb6-4c1b-906c-cb234d52c90e None admin] PATH_INFO: `/v3/services` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:14.752001 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a1d046-ddb6-4c1b-906c-cb234d52c90e None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:07:14.752309 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a1d046-ddb6-4c1b-906c-cb234d52c90e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:14.754251 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a1d046-ddb6-4c1b-906c-cb234d52c90e None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:14.759661 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 410/815] 10.222.0.44 () {56 vars in 1124 bytes} [Wed Jun 29 11:07:14 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:07:14.760486 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a1d046-ddb6-4c1b-906c-cb234d52c90e None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:14.760606 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c6050e-85a5-47cf-a2ae-305a174cfff2 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:14.796341 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc538cc-cad6-47f7-93b1-9d344d1d6cc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:14.797128 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc538cc-cad6-47f7-93b1-9d344d1d6cc1 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:14.797407 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc538cc-cad6-47f7-93b1-9d344d1d6cc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:14.797679 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc538cc-cad6-47f7-93b1-9d344d1d6cc1 None admin] PATH_INFO: `/v3/services/network` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:14.798323 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc538cc-cad6-47f7-93b1-9d344d1d6cc1 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:14.799662 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc538cc-cad6-47f7-93b1-9d344d1d6cc1 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:14.811893 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-2fc538cc-cad6-47f7-93b1-9d344d1d6cc1 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 29 11:07:14.811893 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:07:14.811893 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 29 11:07:14.811893 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 29 11:07:14.811893 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 29 11:07:14.811893 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 29 11:07:14.811893 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 29 11:07:14.811893 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 11:07:14.811893 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 29 11:07:14.811893 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:07:14.811893 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 29 11:07:14.811893 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:07:14.811893 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:07:14.811893 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:07:14.811893 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:07:14.811893 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:07:14.811893 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:07:14.812940 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:07:14.812940 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:07:14.812940 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:07:14.812940 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:07:14.812940 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:07:14.812940 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:07:14.812940 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 29 11:07:14.812940 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 29 11:07:14.812940 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 29 11:07:14.812940 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 29 11:07:14.812940 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:07:14.812940 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:07:14.812940 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:07:14.812940 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:07:14.812940 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:07:14.812940 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:07:14.814088 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:07:14.814088 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jun 29 11:07:14.814088 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:07:14.814088 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:07:14.814088 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:07:14.814088 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:07:14.814088 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:07:14.814088 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:07:14.814088 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:07:14.814088 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:07:14.814088 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 29 11:07:14.814088 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 11:07:14.814088 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 29 11:07:14.814088 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:07:14.814088 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 406/816] 10.222.0.44 () {56 vars in 1111 bytes} [Wed Jun 29 11:07:14 2022] GET /identity/v3/services/network => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:07:14.815099 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc538cc-cad6-47f7-93b1-9d344d1d6cc1 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:14.880879 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a1d046-ddb6-4c1b-906c-cb234d52c90e None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:14.884220 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:14.917446 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46939975-e50e-4a4c-8f2d-3c304026e8bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:14.918624 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46939975-e50e-4a4c-8f2d-3c304026e8bd None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:14.918624 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46939975-e50e-4a4c-8f2d-3c304026e8bd None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:14.918795 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46939975-e50e-4a4c-8f2d-3c304026e8bd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:14.930076 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46939975-e50e-4a4c-8f2d-3c304026e8bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:14.930929 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc538cc-cad6-47f7-93b1-9d344d1d6cc1 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:14.931518 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46939975-e50e-4a4c-8f2d-3c304026e8bd None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:14.949947 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-46939975-e50e-4a4c-8f2d-3c304026e8bd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 11:07:14.949947 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:07:14.949947 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:07:14.949947 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:07:14.949947 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:07:14.949947 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:07:14.949947 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:07:14.949947 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:07:14.949947 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:07:14.949947 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:07:14.949947 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:07:14.949947 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:07:14.949947 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 11:07:14.949947 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 11:07:14.949947 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 11:07:14.950860 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 11:07:14.950860 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:07:14.950860 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:07:14.950860 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:07:14.950860 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:07:14.950860 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:07:14.950860 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:07:14.950860 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:07:14.950860 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jun 29 11:07:14.950860 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:07:14.950860 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:07:14.950860 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:07:14.950860 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:07:14.950860 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:07:14.950860 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:07:14.950860 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:07:14.950860 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:07:14.951754 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 11:07:14.951754 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 11:07:14.951754 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 11:07:14.951754 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jun 29 11:07:14.951754 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 411/817] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jun 29 11:07:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:07:14.952050 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46939975-e50e-4a4c-8f2d-3c304026e8bd None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:14.953524 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:14.973242 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a8aa93-c266-4c8d-bc95-8dca7b55a6f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:14.973848 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a8aa93-c266-4c8d-bc95-8dca7b55a6f5 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:14.974291 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a8aa93-c266-4c8d-bc95-8dca7b55a6f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:14.974574 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a8aa93-c266-4c8d-bc95-8dca7b55a6f5 None admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:14.975758 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a8aa93-c266-4c8d-bc95-8dca7b55a6f5 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:07:14.976103 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a8aa93-c266-4c8d-bc95-8dca7b55a6f5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:14.977519 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a8aa93-c266-4c8d-bc95-8dca7b55a6f5 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:15.000574 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 407/818] 10.222.0.44 () {56 vars in 1133 bytes} [Wed Jun 29 11:07:14 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:07:15.000574 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a8aa93-c266-4c8d-bc95-8dca7b55a6f5 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:15.069704 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46939975-e50e-4a4c-8f2d-3c304026e8bd None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:15.104161 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97422cd1-6c3c-4c84-b7e2-a819ca0ba9cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:15.104822 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97422cd1-6c3c-4c84-b7e2-a819ca0ba9cb None admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:15.105118 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97422cd1-6c3c-4c84-b7e2-a819ca0ba9cb None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:15.105237 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a8aa93-c266-4c8d-bc95-8dca7b55a6f5 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:15.105421 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97422cd1-6c3c-4c84-b7e2-a819ca0ba9cb None admin] PATH_INFO: `/v3/endpoints` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:15.106122 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97422cd1-6c3c-4c84-b7e2-a819ca0ba9cb None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:15.107871 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97422cd1-6c3c-4c84-b7e2-a819ca0ba9cb None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:15.140298 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87edebc-5983-41e3-b2c6-e1b3bc773928 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:15.140959 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87edebc-5983-41e3-b2c6-e1b3bc773928 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:15.142002 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87edebc-5983-41e3-b2c6-e1b3bc773928 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:15.142002 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87edebc-5983-41e3-b2c6-e1b3bc773928 None admin] PATH_INFO: `/v3/services` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:15.142307 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87edebc-5983-41e3-b2c6-e1b3bc773928 None admin] RBAC: Adding query filter params (name=network) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:07:15.142616 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87edebc-5983-41e3-b2c6-e1b3bc773928 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:15.143989 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87edebc-5983-41e3-b2c6-e1b3bc773928 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:15.148974 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 408/819] 10.222.0.44 () {56 vars in 1112 bytes} [Wed Jun 29 11:07:14 2022] GET /identity/v3/services?name=network => generated 124 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:07:15.149728 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87edebc-5983-41e3-b2c6-e1b3bc773928 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:15.191630 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:15.240471 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 412/820] 10.222.0.44 () {60 vars in 1144 bytes} [Wed Jun 29 11:07:14 2022] POST /identity/v3/endpoints => generated 346 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:07:15.241396 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97422cd1-6c3c-4c84-b7e2-a819ca0ba9cb None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:15.249180 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87edebc-5983-41e3-b2c6-e1b3bc773928 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:15.278853 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:15.311599 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4fe9f4-8f67-44b8-94b3-dab6f19871eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:15.313724 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4fe9f4-8f67-44b8-94b3-dab6f19871eb None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:15.314658 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4fe9f4-8f67-44b8-94b3-dab6f19871eb None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:15.315246 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4fe9f4-8f67-44b8-94b3-dab6f19871eb None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:15.316906 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4fe9f4-8f67-44b8-94b3-dab6f19871eb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:07:15.317412 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4fe9f4-8f67-44b8-94b3-dab6f19871eb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:15.319863 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4fe9f4-8f67-44b8-94b3-dab6f19871eb None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:15.336765 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 409/821] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Jun 29 11:07:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:07:15.338285 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4fe9f4-8f67-44b8-94b3-dab6f19871eb None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:15.398394 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97422cd1-6c3c-4c84-b7e2-a819ca0ba9cb None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:15.448905 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d821f08-3e2e-4b8b-9c09-d376accd9999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:15.449884 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d821f08-3e2e-4b8b-9c09-d376accd9999 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:15.450382 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d821f08-3e2e-4b8b-9c09-d376accd9999 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:15.450692 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d821f08-3e2e-4b8b-9c09-d376accd9999 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:15.465813 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:15.472477 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4fe9f4-8f67-44b8-94b3-dab6f19871eb None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:15.519559 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1afe411-3251-45e2-b21b-95bdc376489c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:15.520394 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1afe411-3251-45e2-b21b-95bdc376489c None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:15.521087 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1afe411-3251-45e2-b21b-95bdc376489c None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:15.521367 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1afe411-3251-45e2-b21b-95bdc376489c None admin] PATH_INFO: `/v3/services` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:15.522337 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1afe411-3251-45e2-b21b-95bdc376489c None admin] RBAC: Adding query filter params (type=network) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:07:15.522643 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1afe411-3251-45e2-b21b-95bdc376489c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:15.524907 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1afe411-3251-45e2-b21b-95bdc376489c None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:15.531354 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d821f08-3e2e-4b8b-9c09-d376accd9999 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:15.531513 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 410/822] 10.222.0.44 () {56 vars in 1112 bytes} [Wed Jun 29 11:07:15 2022] GET /identity/v3/services?type=network => generated 350 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:07:15.532289 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1afe411-3251-45e2-b21b-95bdc376489c None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:15.536767 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d821f08-3e2e-4b8b-9c09-d376accd9999 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:15.559150 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:15.656927 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 413/823] 10.222.0.44 () {58 vars in 1282 bytes} [Wed Jun 29 11:07:15 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 29 11:07:15.657762 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d821f08-3e2e-4b8b-9c09-d376accd9999 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:15.674936 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1afe411-3251-45e2-b21b-95bdc376489c None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:15.719047 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e840a7dd-aaf7-4f9c-81eb-1577e5f8052f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:15.719892 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e840a7dd-aaf7-4f9c-81eb-1577e5f8052f None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:15.720403 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e840a7dd-aaf7-4f9c-81eb-1577e5f8052f None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:15.720681 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e840a7dd-aaf7-4f9c-81eb-1577e5f8052f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:15.730623 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e840a7dd-aaf7-4f9c-81eb-1577e5f8052f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:15.732796 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e840a7dd-aaf7-4f9c-81eb-1577e5f8052f None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:15.751697 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-e840a7dd-aaf7-4f9c-81eb-1577e5f8052f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 11:07:15.751697 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:07:15.751697 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:07:15.751697 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:07:15.751697 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:07:15.751697 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:07:15.751697 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:07:15.751697 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:07:15.751697 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:07:15.751697 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:07:15.751697 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:07:15.751697 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:07:15.751697 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 11:07:15.751697 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 11:07:15.751697 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 11:07:15.752666 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 11:07:15.752666 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:07:15.752666 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:07:15.752666 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:07:15.752666 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:07:15.752666 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:07:15.752666 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:07:15.752666 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:07:15.752666 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jun 29 11:07:15.752666 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:07:15.752666 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:07:15.752666 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:07:15.752666 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:07:15.752666 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:07:15.752666 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:07:15.752666 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:07:15.752666 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:07:15.754082 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 11:07:15.754082 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 11:07:15.754082 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 11:07:15.754082 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 11:07:15.754082 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 11:07:15.754082 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 11:07:15.754082 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 11:07:15.754082 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:07:15.754534 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 411/824] 10.222.0.44 () {56 vars in 1111 bytes} [Wed Jun 29 11:07:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:07:15.755077 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e840a7dd-aaf7-4f9c-81eb-1577e5f8052f None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:15.790280 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d821f08-3e2e-4b8b-9c09-d376accd9999 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:15.792086 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:15.806213 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:15.830403 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3d6e3b-2523-4ed3-bb5f-ad39ef90a02d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:15.831163 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3d6e3b-2523-4ed3-bb5f-ad39ef90a02d None admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:15.832412 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3d6e3b-2523-4ed3-bb5f-ad39ef90a02d None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:15.832673 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3d6e3b-2523-4ed3-bb5f-ad39ef90a02d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:15.833479 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3d6e3b-2523-4ed3-bb5f-ad39ef90a02d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:15.834923 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3d6e3b-2523-4ed3-bb5f-ad39ef90a02d None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:15.882094 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e840a7dd-aaf7-4f9c-81eb-1577e5f8052f None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:15.908602 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 414/825] 10.222.0.44 () {60 vars in 1144 bytes} [Wed Jun 29 11:07:15 2022] POST /identity/v3/endpoints => generated 327 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:07:15.911558 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3d6e3b-2523-4ed3-bb5f-ad39ef90a02d None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:15.942515 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66d4555-f03b-4e24-af52-a0b65affbba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:15.943168 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66d4555-f03b-4e24-af52-a0b65affbba1 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:15.943452 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66d4555-f03b-4e24-af52-a0b65affbba1 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:15.943755 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66d4555-f03b-4e24-af52-a0b65affbba1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:15.968816 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66d4555-f03b-4e24-af52-a0b65affbba1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:15.970344 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66d4555-f03b-4e24-af52-a0b65affbba1 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:16.012650 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-a66d4555-f03b-4e24-af52-a0b65affbba1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 11:07:16.012650 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:07:16.012650 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:07:16.012650 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:07:16.012650 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:07:16.012650 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:07:16.012650 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:07:16.012650 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:07:16.012650 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:07:16.012650 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:07:16.012650 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:07:16.012650 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:07:16.012650 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 11:07:16.012650 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 11:07:16.012650 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 11:07:16.013624 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 11:07:16.013624 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:07:16.013624 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:07:16.013624 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:07:16.013624 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:07:16.013624 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:07:16.013624 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:07:16.013624 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:07:16.013624 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jun 29 11:07:16.013624 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:07:16.013624 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:07:16.013624 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:07:16.013624 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:07:16.013624 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:07:16.013624 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:07:16.013624 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:07:16.013624 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:07:16.014802 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 11:07:16.014802 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 11:07:16.014802 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 11:07:16.014802 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:07:16.014802 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 412/826] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jun 29 11:07:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:07:16.019688 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66d4555-f03b-4e24-af52-a0b65affbba1 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:16.048373 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3d6e3b-2523-4ed3-bb5f-ad39ef90a02d None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:16.091291 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a2d2e3-cc52-41ea-a40a-2b1b70c2db99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:16.091985 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a2d2e3-cc52-41ea-a40a-2b1b70c2db99 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:16.092398 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a2d2e3-cc52-41ea-a40a-2b1b70c2db99 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:16.092671 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a2d2e3-cc52-41ea-a40a-2b1b70c2db99 None admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:16.093460 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a2d2e3-cc52-41ea-a40a-2b1b70c2db99 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:07:16.093761 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a2d2e3-cc52-41ea-a40a-2b1b70c2db99 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:16.095445 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a2d2e3-cc52-41ea-a40a-2b1b70c2db99 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:16.112057 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 415/827] 10.222.0.44 () {56 vars in 1148 bytes} [Wed Jun 29 11:07:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:07:16.114252 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a2d2e3-cc52-41ea-a40a-2b1b70c2db99 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:16.140968 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:16.146075 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:16.174777 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66d4555-f03b-4e24-af52-a0b65affbba1 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:16.224085 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00907cb-7c2a-40af-9b69-e25e8e8de689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:16.224807 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00907cb-7c2a-40af-9b69-e25e8e8de689 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:16.225091 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00907cb-7c2a-40af-9b69-e25e8e8de689 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:16.225362 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00907cb-7c2a-40af-9b69-e25e8e8de689 None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:16.226436 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00907cb-7c2a-40af-9b69-e25e8e8de689 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:07:16.226977 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00907cb-7c2a-40af-9b69-e25e8e8de689 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:16.228521 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00907cb-7c2a-40af-9b69-e25e8e8de689 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:16.236327 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a2d2e3-cc52-41ea-a40a-2b1b70c2db99 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:16.251854 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 413/828] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Jun 29 11:07:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 11:07:16.255718 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00907cb-7c2a-40af-9b69-e25e8e8de689 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:16.289739 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e51897-4cc3-4310-beb6-642fe64c75df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:16.290635 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e51897-4cc3-4310-beb6-642fe64c75df None admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:16.290920 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e51897-4cc3-4310-beb6-642fe64c75df None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:16.291195 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e51897-4cc3-4310-beb6-642fe64c75df None admin] PATH_INFO: `/v3/projects/ebe5dffa4bb8440286687daefb6c98f4/users/e3b38493ba904759badc800e396befda/roles/e95c2ce63e114cc19a9dda6ad21e794f` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:16.296208 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e51897-4cc3-4310-beb6-642fe64c75df None admin] RBAC: Authorizing `identity:create_grant(project_id=ebe5dffa4bb8440286687daefb6c98f4, user_id=e3b38493ba904759badc800e396befda, role_id=e95c2ce63e114cc19a9dda6ad21e794f)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:16.297888 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e51897-4cc3-4310-beb6-642fe64c75df None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:16.372781 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 416/829] 10.222.0.44 () {58 vars in 1439 bytes} [Wed Jun 29 11:07:16 2022] PUT /identity/v3/projects/ebe5dffa4bb8440286687daefb6c98f4/users/e3b38493ba904759badc800e396befda/roles/e95c2ce63e114cc19a9dda6ad21e794f => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 11:07:16.381768 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00907cb-7c2a-40af-9b69-e25e8e8de689 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:16.414862 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:16.421633 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593cf2f1-c024-4274-a597-afb2fcff5abe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:16.422439 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593cf2f1-c024-4274-a597-afb2fcff5abe None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:16.422758 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593cf2f1-c024-4274-a597-afb2fcff5abe None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:16.423108 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593cf2f1-c024-4274-a597-afb2fcff5abe None admin] PATH_INFO: `/v3/projects/service` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:16.431655 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593cf2f1-c024-4274-a597-afb2fcff5abe None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:16.433150 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593cf2f1-c024-4274-a597-afb2fcff5abe None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:16.449627 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-593cf2f1-c024-4274-a597-afb2fcff5abe None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 11:07:16.449627 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:07:16.449627 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:07:16.449627 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:07:16.449627 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:07:16.449627 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:07:16.449627 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:07:16.449627 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:07:16.449627 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:07:16.449627 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:07:16.449627 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:07:16.449627 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:07:16.449627 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 11:07:16.449627 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 11:07:16.449627 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 11:07:16.450622 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 11:07:16.450622 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:07:16.450622 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:07:16.450622 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:07:16.450622 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:07:16.450622 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:07:16.450622 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:07:16.450622 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:07:16.450622 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jun 29 11:07:16.450622 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:07:16.450622 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:07:16.450622 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:07:16.450622 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:07:16.450622 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:07:16.450622 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:07:16.450622 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:07:16.450622 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:07:16.451654 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 11:07:16.451654 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 11:07:16.451654 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 11:07:16.451654 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 11:07:16.451654 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 11:07:16.451654 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 11:07:16.451654 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 11:07:16.451654 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:07:16.451654 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 414/830] 10.222.0.44 () {56 vars in 1111 bytes} [Wed Jun 29 11:07:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:07:16.459915 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e51897-4cc3-4310-beb6-642fe64c75df None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:16.565862 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:16.580572 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e51897-4cc3-4310-beb6-642fe64c75df None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:16.617677 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bbcf4f-ffd5-401f-b9b7-ef2ef012c7d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:16.618576 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bbcf4f-ffd5-401f-b9b7-ef2ef012c7d1 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:16.619037 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bbcf4f-ffd5-401f-b9b7-ef2ef012c7d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:16.619312 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bbcf4f-ffd5-401f-b9b7-ef2ef012c7d1 None admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:16.620207 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bbcf4f-ffd5-401f-b9b7-ef2ef012c7d1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:07:16.620518 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bbcf4f-ffd5-401f-b9b7-ef2ef012c7d1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:16.622029 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bbcf4f-ffd5-401f-b9b7-ef2ef012c7d1 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:16.635102 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 417/831] 10.222.0.44 () {56 vars in 1148 bytes} [Wed Jun 29 11:07:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:07:16.640930 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593cf2f1-c024-4274-a597-afb2fcff5abe None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:16.744714 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c744ee96-4f30-49e1-8eac-c043e53d58af None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:16.745228 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c744ee96-4f30-49e1-8eac-c043e53d58af None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:16.745306 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c744ee96-4f30-49e1-8eac-c043e53d58af None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:16.746856 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 418/832] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:07:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:07:16.748508 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593cf2f1-c024-4274-a597-afb2fcff5abe None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:16.752084 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7269343f-853b-4b17-a840-a8253cc42f71 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:16.752084 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7269343f-853b-4b17-a840-a8253cc42f71 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:16.752269 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7269343f-853b-4b17-a840-a8253cc42f71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:16.769205 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-7269343f-853b-4b17-a840-a8253cc42f71 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:07:16.770379 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-7269343f-853b-4b17-a840-a8253cc42f71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:07:16.793770 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef26df8-77d8-4783-aab9-5f20a16459e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:16.795277 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef26df8-77d8-4783-aab9-5f20a16459e8 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:16.795868 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef26df8-77d8-4783-aab9-5f20a16459e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:16.796365 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef26df8-77d8-4783-aab9-5f20a16459e8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:16.798009 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef26df8-77d8-4783-aab9-5f20a16459e8 None admin] RBAC: Adding query filter params (role.id=e95c2ce63e114cc19a9dda6ad21e794f, scope.project.id=ebe5dffa4bb8440286687daefb6c98f4, user.id=796aeeff16df4739bddec1f6dee61bfe) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:07:16.798758 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef26df8-77d8-4783-aab9-5f20a16459e8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:16.801142 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef26df8-77d8-4783-aab9-5f20a16459e8 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:16.807609 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 415/833] 10.222.0.44 () {56 vars in 1374 bytes} [Wed Jun 29 11:07:16 2022] GET /identity/v3/role_assignments?user.id=796aeeff16df4739bddec1f6dee61bfe&scope.project.id=ebe5dffa4bb8440286687daefb6c98f4&role.id=e95c2ce63e114cc19a9dda6ad21e794f => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:07:16.847356 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 419/834] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:07:16 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 29 11:07:16.871741 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef26df8-77d8-4783-aab9-5f20a16459e8 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:16.936841 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:16.990544 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef26df8-77d8-4783-aab9-5f20a16459e8 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:17.026277 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe2e88b-821a-4ef8-97aa-fa43a6bd1dc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:17.027093 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe2e88b-821a-4ef8-97aa-fa43a6bd1dc3 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:17.027379 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe2e88b-821a-4ef8-97aa-fa43a6bd1dc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:17.027664 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe2e88b-821a-4ef8-97aa-fa43a6bd1dc3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:17.040807 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe2e88b-821a-4ef8-97aa-fa43a6bd1dc3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 11:07:17.048628 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-4fe2e88b-821a-4ef8-97aa-fa43a6bd1dc3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 29 11:07:17.048628 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:07:17.048628 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:07:17.048628 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:07:17.048628 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:07:17.048628 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:07:17.048628 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:07:17.048628 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:07:17.048628 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:07:17.048628 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:07:17.048628 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:07:17.048628 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:07:17.048628 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 11:07:17.048628 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 11:07:17.048628 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 11:07:17.048628 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 11:07:17.049552 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 11:07:17.049552 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 11:07:17.049552 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 11:07:17.049552 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 11:07:17.049552 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:07:17.049552 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:07:17.049552 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:07:17.049552 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:07:17.049552 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:07:17.049552 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:07:17.049552 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:07:17.049552 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jun 29 11:07:17.049552 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:07:17.049552 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:07:17.049552 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:07:17.049552 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:07:17.050958 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:07:17.050958 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:07:17.050958 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:07:17.050958 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:07:17.050958 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 11:07:17.050958 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 11:07:17.050958 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 11:07:17.050958 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 11:07:17.050958 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 11:07:17.050958 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 11:07:17.050958 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 29 11:07:17.050958 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:07:17.050958 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 416/835] 10.222.0.44 () {56 vars in 1102 bytes} [Wed Jun 29 11:07:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:07:17.057888 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7269343f-853b-4b17-a840-a8253cc42f71 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:17.151471 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:17.174353 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7269343f-853b-4b17-a840-a8253cc42f71 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:17.210965 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba40609f-f674-48c4-8855-ee8fc88a551c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:17.212012 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba40609f-f674-48c4-8855-ee8fc88a551c None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:17.212485 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba40609f-f674-48c4-8855-ee8fc88a551c None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:17.212986 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba40609f-f674-48c4-8855-ee8fc88a551c None admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:17.213909 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba40609f-f674-48c4-8855-ee8fc88a551c None admin] RBAC: Adding query filter params (name=service) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:07:17.214405 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba40609f-f674-48c4-8855-ee8fc88a551c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:17.215976 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba40609f-f674-48c4-8855-ee8fc88a551c None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:17.228087 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 420/836] 10.222.0.44 () {56 vars in 1103 bytes} [Wed Jun 29 11:07:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 11:07:17.230602 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe2e88b-821a-4ef8-97aa-fa43a6bd1dc3 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:17.336939 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe2e88b-821a-4ef8-97aa-fa43a6bd1dc3 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:17.352103 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:17.374298 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874d7e82-4d11-4993-8c86-f0dcde4af59e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:17.375097 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874d7e82-4d11-4993-8c86-f0dcde4af59e None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:17.375380 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874d7e82-4d11-4993-8c86-f0dcde4af59e None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:17.375653 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874d7e82-4d11-4993-8c86-f0dcde4af59e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:17.386939 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874d7e82-4d11-4993-8c86-f0dcde4af59e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:17.388457 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874d7e82-4d11-4993-8c86-f0dcde4af59e None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:17.406621 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-874d7e82-4d11-4993-8c86-f0dcde4af59e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 11:07:17.406621 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:07:17.406621 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:07:17.406621 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:07:17.406621 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:07:17.406621 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:07:17.406621 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:07:17.406621 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:07:17.406621 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:07:17.406621 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:07:17.406621 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:07:17.406621 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:07:17.406621 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 11:07:17.406621 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 11:07:17.406621 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 11:07:17.407745 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 11:07:17.407745 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:07:17.407745 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:07:17.407745 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:07:17.407745 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:07:17.407745 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:07:17.407745 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:07:17.407745 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:07:17.407745 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jun 29 11:07:17.407745 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:07:17.407745 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:07:17.407745 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:07:17.407745 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:07:17.407745 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:07:17.407745 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:07:17.407745 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:07:17.407745 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:07:17.408713 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 11:07:17.408713 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 11:07:17.408713 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 11:07:17.408713 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:07:17.410520 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 417/837] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jun 29 11:07:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:07:17.416113 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba40609f-f674-48c4-8855-ee8fc88a551c None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:17.429362 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710f8ff6-2cc7-4e74-b9e7-7d072bbefc08 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:17.429629 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710f8ff6-2cc7-4e74-b9e7-7d072bbefc08 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:17.429841 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710f8ff6-2cc7-4e74-b9e7-7d072bbefc08 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:17.431358 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 418/838] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:07:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 29 11:07:17.436622 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eab4c0c-0e9a-4a18-b40e-48fccef4c390 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:17.437193 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eab4c0c-0e9a-4a18-b40e-48fccef4c390 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:17.437789 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eab4c0c-0e9a-4a18-b40e-48fccef4c390 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:17.456004 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-9eab4c0c-0e9a-4a18-b40e-48fccef4c390 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:07:17.458019 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-9eab4c0c-0e9a-4a18-b40e-48fccef4c390 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:07:17.524195 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba40609f-f674-48c4-8855-ee8fc88a551c None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:17.543853 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 419/839] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:07:17 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 11:07:17.560919 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970a9f4e-e335-40ed-a092-b2b9bce5f140 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:17.561849 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eab4c0c-0e9a-4a18-b40e-48fccef4c390 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:17.564394 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970a9f4e-e335-40ed-a092-b2b9bce5f140 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:17.565016 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970a9f4e-e335-40ed-a092-b2b9bce5f140 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:17.565289 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970a9f4e-e335-40ed-a092-b2b9bce5f140 None admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:17.566038 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970a9f4e-e335-40ed-a092-b2b9bce5f140 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:07:17.566343 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970a9f4e-e335-40ed-a092-b2b9bce5f140 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:17.567729 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970a9f4e-e335-40ed-a092-b2b9bce5f140 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:17.577154 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:17.585392 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 421/840] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Jun 29 11:07:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:07:17.586484 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970a9f4e-e335-40ed-a092-b2b9bce5f140 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:17.696002 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970a9f4e-e335-40ed-a092-b2b9bce5f140 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:17.698496 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:17.706243 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eab4c0c-0e9a-4a18-b40e-48fccef4c390 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:17.736981 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17eabb78-ad92-4b39-b589-c38c6dbf5d57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:17.737965 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17eabb78-ad92-4b39-b589-c38c6dbf5d57 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:17.738316 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17eabb78-ad92-4b39-b589-c38c6dbf5d57 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:17.738864 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17eabb78-ad92-4b39-b589-c38c6dbf5d57 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:17.746081 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17eabb78-ad92-4b39-b589-c38c6dbf5d57 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:17.747666 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17eabb78-ad92-4b39-b589-c38c6dbf5d57 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:17.752946 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bd4aa6-0b5d-4467-8283-d8b435cb9d3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:17.754536 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bd4aa6-0b5d-4467-8283-d8b435cb9d3c None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:17.755052 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bd4aa6-0b5d-4467-8283-d8b435cb9d3c None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:17.755797 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bd4aa6-0b5d-4467-8283-d8b435cb9d3c None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:17.756694 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bd4aa6-0b5d-4467-8283-d8b435cb9d3c None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:17.759725 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bd4aa6-0b5d-4467-8283-d8b435cb9d3c None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:17.765061 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-17eabb78-ad92-4b39-b589-c38c6dbf5d57 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 29 11:07:17.765061 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:07:17.765061 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:07:17.765061 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:07:17.765061 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:07:17.765061 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:07:17.765061 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:07:17.765061 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:07:17.765061 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:07:17.765061 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:07:17.765061 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:07:17.765061 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:07:17.765061 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 11:07:17.765061 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 11:07:17.765061 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 11:07:17.765061 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 11:07:17.766012 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:07:17.766012 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:07:17.766012 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 11:07:17.766012 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 11:07:17.766012 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 11:07:17.766012 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 11:07:17.766012 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:07:17.766012 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:07:17.766012 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:07:17.766012 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:07:17.766012 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:07:17.766012 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jun 29 11:07:17.766012 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:07:17.766012 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:07:17.766012 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:07:17.766012 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:07:17.766012 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:07:17.766012 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:07:17.767035 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:07:17.767035 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:07:17.767035 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 11:07:17.767035 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 11:07:17.767035 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 11:07:17.767035 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 11:07:17.767035 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 11:07:17.767035 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 11:07:17.767035 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 29 11:07:17.767035 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jun 29 11:07:17.771289 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 422/841] 10.222.0.44 () {56 vars in 1117 bytes} [Wed Jun 29 11:07:17 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:07:17.774730 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17eabb78-ad92-4b39-b589-c38c6dbf5d57 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:17.780076 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-c5bd4aa6-0b5d-4467-8283-d8b435cb9d3c None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 29 11:07:17.780076 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:07:17.780076 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 29 11:07:17.780076 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 29 11:07:17.780076 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 29 11:07:17.780076 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 29 11:07:17.780076 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 29 11:07:17.780076 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 11:07:17.780076 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 29 11:07:17.780076 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:07:17.780076 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 29 11:07:17.780076 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:07:17.780076 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:07:17.780076 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:07:17.780076 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:07:17.780076 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:07:17.780076 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:07:17.781062 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:07:17.781062 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:07:17.781062 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:07:17.781062 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:07:17.781062 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:07:17.781062 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:07:17.781062 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 29 11:07:17.781062 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 29 11:07:17.781062 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 29 11:07:17.781062 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 29 11:07:17.781062 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:07:17.781062 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:07:17.781062 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:07:17.781062 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:07:17.781062 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:07:17.781062 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:07:17.781910 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:07:17.781910 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jun 29 11:07:17.781910 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:07:17.781910 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:07:17.781910 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:07:17.781910 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:07:17.781910 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:07:17.781910 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:07:17.781910 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:07:17.781910 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:07:17.781910 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 29 11:07:17.781910 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 11:07:17.781910 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 29 11:07:17.781910 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:07:17.782661 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 420/842] 10.222.0.44 () {56 vars in 1114 bytes} [Wed Jun 29 11:07:17 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 11:07:17.789929 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bd4aa6-0b5d-4467-8283-d8b435cb9d3c None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:17.842651 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:17.896023 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17eabb78-ad92-4b39-b589-c38c6dbf5d57 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:17.920925 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bd4aa6-0b5d-4467-8283-d8b435cb9d3c None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:17.931220 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71af4df1-b6c1-4531-a9b5-d4d3d8d53f13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:17.931973 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71af4df1-b6c1-4531-a9b5-d4d3d8d53f13 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:17.932472 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71af4df1-b6c1-4531-a9b5-d4d3d8d53f13 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:17.932842 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71af4df1-b6c1-4531-a9b5-d4d3d8d53f13 None admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:17.934026 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71af4df1-b6c1-4531-a9b5-d4d3d8d53f13 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:07:17.934238 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71af4df1-b6c1-4531-a9b5-d4d3d8d53f13 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:17.935818 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71af4df1-b6c1-4531-a9b5-d4d3d8d53f13 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:17.959404 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 423/843] 10.222.0.44 () {56 vars in 1149 bytes} [Wed Jun 29 11:07:17 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:07:17.960395 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71af4df1-b6c1-4531-a9b5-d4d3d8d53f13 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:17.967193 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0415445-1c6a-4b82-95f9-b6f91b6d4ab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:17.968095 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0415445-1c6a-4b82-95f9-b6f91b6d4ab0 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:17.968530 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0415445-1c6a-4b82-95f9-b6f91b6d4ab0 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:17.968971 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0415445-1c6a-4b82-95f9-b6f91b6d4ab0 None admin] PATH_INFO: `/v3/services` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:17.970354 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0415445-1c6a-4b82-95f9-b6f91b6d4ab0 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:07:17.970833 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0415445-1c6a-4b82-95f9-b6f91b6d4ab0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:17.972858 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0415445-1c6a-4b82-95f9-b6f91b6d4ab0 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:17.984312 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 421/844] 10.222.0.44 () {56 vars in 1114 bytes} [Wed Jun 29 11:07:17 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:07:17.985880 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0415445-1c6a-4b82-95f9-b6f91b6d4ab0 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:18.016195 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:18.085629 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71af4df1-b6c1-4531-a9b5-d4d3d8d53f13 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:18.106026 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:18.116101 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0415445-1c6a-4b82-95f9-b6f91b6d4ab0 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:18.125176 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc43effc-c40c-4d57-8e8d-8e9f48c5f7ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:18.126173 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc43effc-c40c-4d57-8e8d-8e9f48c5f7ed None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:18.126740 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc43effc-c40c-4d57-8e8d-8e9f48c5f7ed None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:18.127435 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc43effc-c40c-4d57-8e8d-8e9f48c5f7ed None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:18.155264 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418d4800-0771-43f1-98b7-19a70ccfd57e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:18.155989 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418d4800-0771-43f1-98b7-19a70ccfd57e None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:18.156412 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418d4800-0771-43f1-98b7-19a70ccfd57e None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:18.156650 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418d4800-0771-43f1-98b7-19a70ccfd57e None admin] PATH_INFO: `/v3/services` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:18.157358 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418d4800-0771-43f1-98b7-19a70ccfd57e None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:07:18.157714 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418d4800-0771-43f1-98b7-19a70ccfd57e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:18.159123 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418d4800-0771-43f1-98b7-19a70ccfd57e None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:18.172382 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 422/845] 10.222.0.44 () {56 vars in 1114 bytes} [Wed Jun 29 11:07:17 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 11:07:18.217309 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc43effc-c40c-4d57-8e8d-8e9f48c5f7ed None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:18.218757 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc43effc-c40c-4d57-8e8d-8e9f48c5f7ed None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:18.328646 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 424/846] 10.222.0.44 () {58 vars in 1282 bytes} [Wed Jun 29 11:07:17 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 29 11:07:18.334415 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418d4800-0771-43f1-98b7-19a70ccfd57e None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:18.446658 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:18.460475 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418d4800-0771-43f1-98b7-19a70ccfd57e None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:18.498624 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04a9f66-f240-415f-976f-15cac12acde5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:18.499786 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04a9f66-f240-415f-976f-15cac12acde5 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:18.500315 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04a9f66-f240-415f-976f-15cac12acde5 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:18.500794 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04a9f66-f240-415f-976f-15cac12acde5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:18.512014 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04a9f66-f240-415f-976f-15cac12acde5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:18.513721 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04a9f66-f240-415f-976f-15cac12acde5 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:18.533007 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-c04a9f66-f240-415f-976f-15cac12acde5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 11:07:18.533007 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:07:18.533007 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:07:18.533007 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:07:18.533007 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:07:18.533007 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:07:18.533007 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:07:18.533007 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:07:18.533007 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:07:18.533007 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:07:18.533007 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:07:18.533007 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:07:18.533007 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 11:07:18.533007 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 11:07:18.533007 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 11:07:18.534021 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 11:07:18.534021 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:07:18.534021 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:07:18.534021 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:07:18.534021 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:07:18.534021 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:07:18.534021 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:07:18.534021 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:07:18.534021 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jun 29 11:07:18.534021 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:07:18.534021 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:07:18.534021 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:07:18.534021 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:07:18.534021 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:07:18.534021 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:07:18.534021 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:07:18.534021 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:07:18.535516 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 11:07:18.535516 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 11:07:18.535516 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 11:07:18.535516 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:07:18.537839 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 423/847] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jun 29 11:07:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:07:18.545243 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc43effc-c40c-4d57-8e8d-8e9f48c5f7ed None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:18.562029 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:18.576251 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd3ef0f-4276-4d70-bb50-2957ae78a65c None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:18.576778 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd3ef0f-4276-4d70-bb50-2957ae78a65c None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:18.577201 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd3ef0f-4276-4d70-bb50-2957ae78a65c None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:18.578954 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 424/848] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:07:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 29 11:07:18.585447 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaa194b-d48a-4dd4-9c25-0c9fe6785c92 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:18.585930 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaa194b-d48a-4dd4-9c25-0c9fe6785c92 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:18.586352 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaa194b-d48a-4dd4-9c25-0c9fe6785c92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:18.605020 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-bbaa194b-d48a-4dd4-9c25-0c9fe6785c92 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:07:18.606555 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-bbaa194b-d48a-4dd4-9c25-0c9fe6785c92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:07:18.658216 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc43effc-c40c-4d57-8e8d-8e9f48c5f7ed None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:18.687416 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 425/849] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:07:18 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 29 11:07:18.698621 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb5e225-5c0d-4d48-a084-35e141e637df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:18.699499 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb5e225-5c0d-4d48-a084-35e141e637df None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:18.699953 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb5e225-5c0d-4d48-a084-35e141e637df None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:18.700394 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb5e225-5c0d-4d48-a084-35e141e637df None admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:18.701250 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb5e225-5c0d-4d48-a084-35e141e637df None admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:07:18.701694 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb5e225-5c0d-4d48-a084-35e141e637df None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:18.703206 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb5e225-5c0d-4d48-a084-35e141e637df None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:18.707903 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbaa194b-d48a-4dd4-9c25-0c9fe6785c92 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:18.724179 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 425/850] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Jun 29 11:07:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:07:18.726034 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb5e225-5c0d-4d48-a084-35e141e637df None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:18.797085 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:18.823909 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:18.838222 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbaa194b-d48a-4dd4-9c25-0c9fe6785c92 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:18.852817 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb5e225-5c0d-4d48-a084-35e141e637df None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:18.873717 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc1261a-ddd6-457f-bb3e-649245467b58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:18.874519 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc1261a-ddd6-457f-bb3e-649245467b58 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:18.874849 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc1261a-ddd6-457f-bb3e-649245467b58 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:18.875119 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc1261a-ddd6-457f-bb3e-649245467b58 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:18.886273 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc1261a-ddd6-457f-bb3e-649245467b58 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 11:07:18.895426 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-bbc1261a-ddd6-457f-bb3e-649245467b58 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 11:07:18.895426 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:07:18.895426 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:07:18.895426 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:07:18.895426 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:07:18.895426 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:07:18.895426 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:07:18.895426 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:07:18.895426 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:07:18.895426 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:07:18.895426 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:07:18.895426 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:07:18.895426 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 11:07:18.895426 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 11:07:18.895426 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 11:07:18.895426 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 11:07:18.896378 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 11:07:18.896378 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 11:07:18.896378 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 11:07:18.896378 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 11:07:18.896378 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:07:18.896378 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:07:18.896378 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:07:18.896378 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:07:18.896378 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:07:18.896378 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:07:18.896378 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:07:18.896378 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jun 29 11:07:18.896378 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:07:18.896378 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:07:18.896378 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:07:18.896378 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:07:18.897226 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:07:18.897226 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:07:18.897226 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:07:18.897226 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:07:18.897226 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 11:07:18.897226 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 11:07:18.897226 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 11:07:18.897226 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 11:07:18.897226 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 11:07:18.897226 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 11:07:18.897226 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 11:07:18.897226 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:07:18.899085 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d07848-9531-4759-8626-144285b8a6aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:18.900134 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d07848-9531-4759-8626-144285b8a6aa None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:18.900621 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d07848-9531-4759-8626-144285b8a6aa None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:18.901253 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d07848-9531-4759-8626-144285b8a6aa None admin] PATH_INFO: `/v3/projects/service` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:18.901858 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 426/851] 10.222.0.44 () {56 vars in 1096 bytes} [Wed Jun 29 11:07:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 11:07:18.902943 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc1261a-ddd6-457f-bb3e-649245467b58 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:18.919934 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d07848-9531-4759-8626-144285b8a6aa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:18.922118 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d07848-9531-4759-8626-144285b8a6aa None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:18.942459 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-73d07848-9531-4759-8626-144285b8a6aa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 11:07:18.942459 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:07:18.942459 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:07:18.942459 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:07:18.942459 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:07:18.942459 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:07:18.942459 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:07:18.942459 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:07:18.942459 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:07:18.942459 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:07:18.942459 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:07:18.942459 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:07:18.942459 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 11:07:18.942459 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 11:07:18.942459 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 11:07:18.943379 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 11:07:18.943379 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:07:18.943379 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:07:18.943379 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:07:18.943379 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:07:18.943379 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:07:18.943379 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:07:18.943379 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:07:18.943379 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jun 29 11:07:18.943379 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:07:18.943379 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:07:18.943379 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:07:18.943379 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:07:18.943379 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:07:18.943379 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:07:18.943379 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:07:18.943379 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:07:18.944333 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 11:07:18.944333 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 11:07:18.944333 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 11:07:18.944333 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 11:07:18.944333 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 11:07:18.944333 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 11:07:18.944333 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 11:07:18.944333 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jun 29 11:07:18.948182 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 426/852] 10.222.0.44 () {56 vars in 1111 bytes} [Wed Jun 29 11:07:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:07:18.951174 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d07848-9531-4759-8626-144285b8a6aa None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:19.022455 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc1261a-ddd6-457f-bb3e-649245467b58 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:19.051991 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:19.059728 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07b148d-1a47-452b-9576-58e4debf5613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:19.060609 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07b148d-1a47-452b-9576-58e4debf5613 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:19.060706 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07b148d-1a47-452b-9576-58e4debf5613 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:19.060960 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07b148d-1a47-452b-9576-58e4debf5613 None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:19.061817 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07b148d-1a47-452b-9576-58e4debf5613 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:07:19.062125 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07b148d-1a47-452b-9576-58e4debf5613 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:19.063618 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07b148d-1a47-452b-9576-58e4debf5613 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:19.075631 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 427/853] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Jun 29 11:07:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:07:19.077067 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405c6d3d-2595-40a0-bafa-e6f73912375b None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:19.077426 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405c6d3d-2595-40a0-bafa-e6f73912375b None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:19.077710 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405c6d3d-2595-40a0-bafa-e6f73912375b None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:19.079969 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 428/854] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:07:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:07:19.081043 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405c6d3d-2595-40a0-bafa-e6f73912375b None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:19.091488 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d07848-9531-4759-8626-144285b8a6aa None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:19.126323 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ef2737-c805-413d-8beb-fb1b53e9c14c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:19.126983 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ef2737-c805-413d-8beb-fb1b53e9c14c None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:19.127268 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ef2737-c805-413d-8beb-fb1b53e9c14c None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:19.127544 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ef2737-c805-413d-8beb-fb1b53e9c14c None admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:19.129220 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ef2737-c805-413d-8beb-fb1b53e9c14c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:07:19.129220 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ef2737-c805-413d-8beb-fb1b53e9c14c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:19.130632 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ef2737-c805-413d-8beb-fb1b53e9c14c None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:19.137912 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:19.148646 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 427/855] 10.222.0.44 () {56 vars in 1148 bytes} [Wed Jun 29 11:07:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 11:07:19.148646 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6127e22-52af-4835-97b1-a64828e5d03d None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:19.148646 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6127e22-52af-4835-97b1-a64828e5d03d None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:19.148646 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6127e22-52af-4835-97b1-a64828e5d03d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:19.163448 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-f6127e22-52af-4835-97b1-a64828e5d03d None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:07:19.164687 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-f6127e22-52af-4835-97b1-a64828e5d03d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:07:19.197175 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405c6d3d-2595-40a0-bafa-e6f73912375b None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:19.231732 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1143f2b-6dac-4c5c-89e5-b6c598dd5fe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:19.232629 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1143f2b-6dac-4c5c-89e5-b6c598dd5fe0 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:19.232922 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1143f2b-6dac-4c5c-89e5-b6c598dd5fe0 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:19.233250 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1143f2b-6dac-4c5c-89e5-b6c598dd5fe0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:19.239905 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 428/856] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:07:19 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 11:07:19.239905 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6127e22-52af-4835-97b1-a64828e5d03d None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:19.249857 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1143f2b-6dac-4c5c-89e5-b6c598dd5fe0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:19.251963 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1143f2b-6dac-4c5c-89e5-b6c598dd5fe0 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:19.274595 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-f1143f2b-6dac-4c5c-89e5-b6c598dd5fe0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 11:07:19.274595 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:07:19.274595 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:07:19.274595 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:07:19.274595 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:07:19.274595 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:07:19.274595 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:07:19.274595 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:07:19.274595 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:07:19.274595 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:07:19.274595 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:07:19.274595 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:07:19.274595 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 11:07:19.274595 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 11:07:19.274595 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 11:07:19.275516 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 11:07:19.275516 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:07:19.275516 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:07:19.275516 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:07:19.275516 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:07:19.275516 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:07:19.275516 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:07:19.275516 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:07:19.275516 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jun 29 11:07:19.275516 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:07:19.275516 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:07:19.275516 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:07:19.275516 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:07:19.275516 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:07:19.275516 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:07:19.275516 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:07:19.275516 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:07:19.276446 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 11:07:19.276446 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 11:07:19.276446 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 11:07:19.276446 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:07:19.276446 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 429/857] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jun 29 11:07:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:07:19.277119 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1143f2b-6dac-4c5c-89e5-b6c598dd5fe0 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:19.327863 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:19.354267 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6127e22-52af-4835-97b1-a64828e5d03d None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:19.382851 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:19.386852 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1143f2b-6dac-4c5c-89e5-b6c598dd5fe0 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:19.392046 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d66921-7028-4b89-b229-948af0b7aa99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:19.392731 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d66921-7028-4b89-b229-948af0b7aa99 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:19.393018 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d66921-7028-4b89-b229-948af0b7aa99 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:19.393299 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d66921-7028-4b89-b229-948af0b7aa99 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:19.394212 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d66921-7028-4b89-b229-948af0b7aa99 None admin] RBAC: Adding query filter params (role.id=a8212be3426741cbb1b38b1646c66405, scope.project.id=ebe5dffa4bb8440286687daefb6c98f4, user.id=55487c2107a44c15b656b4865ee0d4fa) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:07:19.394547 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d66921-7028-4b89-b229-948af0b7aa99 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:19.396075 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d66921-7028-4b89-b229-948af0b7aa99 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:19.402503 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 429/858] 10.222.0.44 () {56 vars in 1374 bytes} [Wed Jun 29 11:07:19 2022] GET /identity/v3/role_assignments?user.id=55487c2107a44c15b656b4865ee0d4fa&scope.project.id=ebe5dffa4bb8440286687daefb6c98f4&role.id=a8212be3426741cbb1b38b1646c66405 => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:07:19.403343 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d66921-7028-4b89-b229-948af0b7aa99 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:19.445220 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7f3fa5-3b6c-4107-98f5-b40f22346f6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:19.447677 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7f3fa5-3b6c-4107-98f5-b40f22346f6a None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:19.448281 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7f3fa5-3b6c-4107-98f5-b40f22346f6a None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:19.448755 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7f3fa5-3b6c-4107-98f5-b40f22346f6a None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:19.449620 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7f3fa5-3b6c-4107-98f5-b40f22346f6a None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:19.451220 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7f3fa5-3b6c-4107-98f5-b40f22346f6a None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:19.469820 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-9a7f3fa5-3b6c-4107-98f5-b40f22346f6a None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 29 11:07:19.469820 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:07:19.469820 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 29 11:07:19.469820 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 29 11:07:19.469820 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 29 11:07:19.469820 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 29 11:07:19.469820 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 29 11:07:19.469820 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 11:07:19.469820 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 29 11:07:19.469820 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:07:19.469820 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 29 11:07:19.469820 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:07:19.469820 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:07:19.469820 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:07:19.469820 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:07:19.469820 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:07:19.470880 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:07:19.470880 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:07:19.470880 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:07:19.470880 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:07:19.470880 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:07:19.470880 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:07:19.470880 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:07:19.470880 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 29 11:07:19.470880 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 29 11:07:19.470880 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 29 11:07:19.470880 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 29 11:07:19.470880 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:07:19.470880 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:07:19.470880 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:07:19.470880 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:07:19.470880 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:07:19.471725 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:07:19.471725 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:07:19.471725 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jun 29 11:07:19.471725 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:07:19.471725 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:07:19.471725 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:07:19.471725 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:07:19.471725 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:07:19.471725 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:07:19.471725 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:07:19.471725 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:07:19.471725 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 29 11:07:19.471725 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 11:07:19.471725 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 29 11:07:19.471725 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:07:19.474205 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 430/859] 10.222.0.44 () {56 vars in 1132 bytes} [Wed Jun 29 11:07:19 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:07:19.479343 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7f3fa5-3b6c-4107-98f5-b40f22346f6a None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:19.536039 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d66921-7028-4b89-b229-948af0b7aa99 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:19.587503 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380e0ce7-ff75-4c41-8c7e-8a8a71fb3f6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:19.588532 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380e0ce7-ff75-4c41-8c7e-8a8a71fb3f6d None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:19.589018 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380e0ce7-ff75-4c41-8c7e-8a8a71fb3f6d None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:19.589506 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380e0ce7-ff75-4c41-8c7e-8a8a71fb3f6d None admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:19.590232 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380e0ce7-ff75-4c41-8c7e-8a8a71fb3f6d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:07:19.590539 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380e0ce7-ff75-4c41-8c7e-8a8a71fb3f6d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:19.591964 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380e0ce7-ff75-4c41-8c7e-8a8a71fb3f6d None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:19.608149 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7f3fa5-3b6c-4107-98f5-b40f22346f6a None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:19.609954 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 430/860] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Jun 29 11:07:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:07:19.616161 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380e0ce7-ff75-4c41-8c7e-8a8a71fb3f6d None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:19.619584 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:19.649053 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cf78bc-6a0c-4473-993c-daf8fc3c345b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:19.649736 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cf78bc-6a0c-4473-993c-daf8fc3c345b None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:19.650008 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cf78bc-6a0c-4473-993c-daf8fc3c345b None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:19.650273 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cf78bc-6a0c-4473-993c-daf8fc3c345b None admin] PATH_INFO: `/v3/services` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:19.651063 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cf78bc-6a0c-4473-993c-daf8fc3c345b None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:07:19.651293 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cf78bc-6a0c-4473-993c-daf8fc3c345b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:19.653382 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cf78bc-6a0c-4473-993c-daf8fc3c345b None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:19.661181 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 431/861] 10.222.0.44 () {56 vars in 1126 bytes} [Wed Jun 29 11:07:19 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 11:07:19.667868 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cf78bc-6a0c-4473-993c-daf8fc3c345b None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:19.686087 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:19.736383 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380e0ce7-ff75-4c41-8c7e-8a8a71fb3f6d None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:19.774063 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e44928-a86e-4a4f-8a72-d477e86a7964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:19.774993 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e44928-a86e-4a4f-8a72-d477e86a7964 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:19.775348 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e44928-a86e-4a4f-8a72-d477e86a7964 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:19.775697 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e44928-a86e-4a4f-8a72-d477e86a7964 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:19.783378 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e44928-a86e-4a4f-8a72-d477e86a7964 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:19.785280 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e44928-a86e-4a4f-8a72-d477e86a7964 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:19.800904 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cf78bc-6a0c-4473-993c-daf8fc3c345b None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:19.801871 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-82e44928-a86e-4a4f-8a72-d477e86a7964 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 29 11:07:19.801871 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:07:19.801871 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:07:19.801871 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:07:19.801871 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:07:19.801871 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:07:19.801871 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:07:19.801871 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:07:19.801871 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:07:19.801871 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:07:19.801871 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:07:19.801871 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:07:19.801871 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 11:07:19.801871 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 11:07:19.801871 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 11:07:19.801871 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 11:07:19.802881 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:07:19.802881 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:07:19.802881 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 11:07:19.802881 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 11:07:19.802881 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 11:07:19.802881 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 11:07:19.802881 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:07:19.802881 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:07:19.802881 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:07:19.802881 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:07:19.802881 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:07:19.802881 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jun 29 11:07:19.802881 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:07:19.802881 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:07:19.802881 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:07:19.802881 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:07:19.802881 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:07:19.802881 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:07:19.803858 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:07:19.803858 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:07:19.803858 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 11:07:19.803858 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 11:07:19.803858 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 11:07:19.803858 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 11:07:19.803858 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 11:07:19.803858 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 11:07:19.803858 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 29 11:07:19.803858 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jun 29 11:07:19.803858 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 431/862] 10.222.0.44 () {56 vars in 1096 bytes} [Wed Jun 29 11:07:19 2022] GET /identity/v3/users/swift => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 11:07:19.808902 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e44928-a86e-4a4f-8a72-d477e86a7964 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:19.845357 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abac71ee-d50b-403a-9501-b592b9a5eb36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:19.846832 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abac71ee-d50b-403a-9501-b592b9a5eb36 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:19.847140 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abac71ee-d50b-403a-9501-b592b9a5eb36 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:19.847423 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abac71ee-d50b-403a-9501-b592b9a5eb36 None admin] PATH_INFO: `/v3/services` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:19.848173 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abac71ee-d50b-403a-9501-b592b9a5eb36 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:07:19.848486 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abac71ee-d50b-403a-9501-b592b9a5eb36 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:19.849863 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abac71ee-d50b-403a-9501-b592b9a5eb36 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:19.858860 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 432/863] 10.222.0.44 () {56 vars in 1126 bytes} [Wed Jun 29 11:07:19 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:07:19.924856 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e44928-a86e-4a4f-8a72-d477e86a7964 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:19.940710 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:19.965183 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc1b845-d52d-4982-a816-aff049e0d3df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:19.965950 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc1b845-d52d-4982-a816-aff049e0d3df None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:19.966233 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc1b845-d52d-4982-a816-aff049e0d3df None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:19.966512 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc1b845-d52d-4982-a816-aff049e0d3df None admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:19.967374 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc1b845-d52d-4982-a816-aff049e0d3df None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:07:19.967689 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc1b845-d52d-4982-a816-aff049e0d3df None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:19.969212 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc1b845-d52d-4982-a816-aff049e0d3df None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:19.990949 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 432/864] 10.222.0.44 () {56 vars in 1135 bytes} [Wed Jun 29 11:07:19 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:07:19.996941 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abac71ee-d50b-403a-9501-b592b9a5eb36 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:20.038079 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:20.105088 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abac71ee-d50b-403a-9501-b592b9a5eb36 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:20.139526 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2cccb9-c6a5-40f5-9458-453c5bd7d348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:20.140224 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2cccb9-c6a5-40f5-9458-453c5bd7d348 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:20.140500 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2cccb9-c6a5-40f5-9458-453c5bd7d348 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:20.140780 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2cccb9-c6a5-40f5-9458-453c5bd7d348 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:20.213177 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2cccb9-c6a5-40f5-9458-453c5bd7d348 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:20.215023 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2cccb9-c6a5-40f5-9458-453c5bd7d348 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:20.268235 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:20.323632 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 433/865] 10.222.0.44 () {58 vars in 1282 bytes} [Wed Jun 29 11:07:19 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 29 11:07:20.329032 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc1b845-d52d-4982-a816-aff049e0d3df None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:20.358294 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57443328-de16-4e0e-87a1-cc930bb2a2ba None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:20.358893 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57443328-de16-4e0e-87a1-cc930bb2a2ba None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:20.359421 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57443328-de16-4e0e-87a1-cc930bb2a2ba None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:20.361025 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 434/866] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:07:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 29 11:07:20.366004 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fbd634-594c-4c77-8e3b-85965a472821 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:20.366509 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fbd634-594c-4c77-8e3b-85965a472821 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:20.366926 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fbd634-594c-4c77-8e3b-85965a472821 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:20.384808 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-06fbd634-594c-4c77-8e3b-85965a472821 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:07:20.386237 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-06fbd634-594c-4c77-8e3b-85965a472821 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:07:20.440019 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc1b845-d52d-4982-a816-aff049e0d3df None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:20.458045 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 435/867] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:07:20 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 29 11:07:20.476298 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55f4d27-e136-45e5-afee-b65069339b5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:20.477092 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55f4d27-e136-45e5-afee-b65069339b5c None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:20.477373 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55f4d27-e136-45e5-afee-b65069339b5c None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:20.477700 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55f4d27-e136-45e5-afee-b65069339b5c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:20.480462 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fbd634-594c-4c77-8e3b-85965a472821 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:20.488127 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55f4d27-e136-45e5-afee-b65069339b5c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:20.489833 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55f4d27-e136-45e5-afee-b65069339b5c None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:20.513406 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-c55f4d27-e136-45e5-afee-b65069339b5c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 11:07:20.513406 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:07:20.513406 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:07:20.513406 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:07:20.513406 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:07:20.513406 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:07:20.513406 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:07:20.513406 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:07:20.513406 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:07:20.513406 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:07:20.513406 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:07:20.513406 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:07:20.513406 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 11:07:20.513406 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 11:07:20.513406 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 11:07:20.514345 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 11:07:20.514345 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:07:20.514345 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:07:20.514345 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:07:20.514345 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:07:20.514345 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:07:20.514345 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:07:20.514345 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:07:20.514345 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jun 29 11:07:20.514345 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:07:20.514345 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:07:20.514345 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:07:20.514345 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:07:20.514345 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:07:20.514345 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:07:20.514345 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:07:20.514345 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:07:20.515229 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 11:07:20.515229 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 11:07:20.515229 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 11:07:20.515229 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jun 29 11:07:20.516238 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 433/868] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jun 29 11:07:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:07:20.522139 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55f4d27-e136-45e5-afee-b65069339b5c None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:20.552321 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:20.559081 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:20.591613 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fbd634-594c-4c77-8e3b-85965a472821 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:20.631607 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c503c5-e31d-41aa-b56f-16bbe46538a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:20.632620 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c503c5-e31d-41aa-b56f-16bbe46538a4 None admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:20.633052 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c503c5-e31d-41aa-b56f-16bbe46538a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:20.633502 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c503c5-e31d-41aa-b56f-16bbe46538a4 None admin] PATH_INFO: `/v3/services` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:20.634400 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c503c5-e31d-41aa-b56f-16bbe46538a4 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:20.636005 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c503c5-e31d-41aa-b56f-16bbe46538a4 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:20.707168 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 436/869] 10.222.0.44 () {60 vars in 1141 bytes} [Wed Jun 29 11:07:20 2022] POST /identity/v3/services => generated 251 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 11:07:20.789459 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55f4d27-e136-45e5-afee-b65069339b5c None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:20.833868 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3125121-c7d4-4be6-81d2-d974844cb624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:20.834545 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3125121-c7d4-4be6-81d2-d974844cb624 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:20.834825 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3125121-c7d4-4be6-81d2-d974844cb624 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:20.835095 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3125121-c7d4-4be6-81d2-d974844cb624 None admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:20.835821 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3125121-c7d4-4be6-81d2-d974844cb624 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:07:20.836158 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3125121-c7d4-4be6-81d2-d974844cb624 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:20.837539 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3125121-c7d4-4be6-81d2-d974844cb624 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:20.850840 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 434/870] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Jun 29 11:07:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 11:07:20.856305 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c503c5-e31d-41aa-b56f-16bbe46538a4 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:20.960327 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c503c5-e31d-41aa-b56f-16bbe46538a4 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:20.985266 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:20.993864 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73ac102-2fae-408d-9642-500ddecaaade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:20.994719 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73ac102-2fae-408d-9642-500ddecaaade None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:20.995000 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73ac102-2fae-408d-9642-500ddecaaade None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:20.995277 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73ac102-2fae-408d-9642-500ddecaaade None admin] PATH_INFO: `/v3/projects/service` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:21.001911 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73ac102-2fae-408d-9642-500ddecaaade None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:21.003856 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73ac102-2fae-408d-9642-500ddecaaade None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:21.017822 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-c73ac102-2fae-408d-9642-500ddecaaade None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 11:07:21.017822 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:07:21.017822 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:07:21.017822 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:07:21.017822 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:07:21.017822 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:07:21.017822 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:07:21.017822 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:07:21.017822 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:07:21.017822 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:07:21.017822 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:07:21.017822 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:07:21.017822 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 11:07:21.017822 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 11:07:21.017822 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 11:07:21.018722 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 11:07:21.018722 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:07:21.018722 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:07:21.018722 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:07:21.018722 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:07:21.018722 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:07:21.018722 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:07:21.018722 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:07:21.018722 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jun 29 11:07:21.018722 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:07:21.018722 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:07:21.018722 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:07:21.018722 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:07:21.018722 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:07:21.018722 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:07:21.018722 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:07:21.018722 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:07:21.020213 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 11:07:21.020213 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 11:07:21.020213 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 11:07:21.020213 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 11:07:21.020213 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 11:07:21.020213 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 11:07:21.020213 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 11:07:21.020213 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:07:21.020213 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 437/871] 10.222.0.44 () {56 vars in 1111 bytes} [Wed Jun 29 11:07:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:07:21.025532 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3125121-c7d4-4be6-81d2-d974844cb624 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:21.107522 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:21.129260 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3125121-c7d4-4be6-81d2-d974844cb624 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:21.163800 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3607f32b-533e-4694-90b4-2058dcf5d21d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:21.164484 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3607f32b-533e-4694-90b4-2058dcf5d21d None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:21.164764 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3607f32b-533e-4694-90b4-2058dcf5d21d None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:21.165038 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3607f32b-533e-4694-90b4-2058dcf5d21d None admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:21.165817 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3607f32b-533e-4694-90b4-2058dcf5d21d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:07:21.166123 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3607f32b-533e-4694-90b4-2058dcf5d21d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:21.167542 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3607f32b-533e-4694-90b4-2058dcf5d21d None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:21.179587 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 435/872] 10.222.0.44 () {56 vars in 1148 bytes} [Wed Jun 29 11:07:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:07:21.185364 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73ac102-2fae-408d-9642-500ddecaaade None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:21.288995 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73ac102-2fae-408d-9642-500ddecaaade None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:21.323202 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2e29b2-9ad2-47f3-bb21-e4f5a957011c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:21.323871 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2e29b2-9ad2-47f3-bb21-e4f5a957011c None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:21.324159 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2e29b2-9ad2-47f3-bb21-e4f5a957011c None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:21.324431 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2e29b2-9ad2-47f3-bb21-e4f5a957011c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:21.325256 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2e29b2-9ad2-47f3-bb21-e4f5a957011c None admin] RBAC: Adding query filter params (role.id=e95c2ce63e114cc19a9dda6ad21e794f, scope.project.id=ebe5dffa4bb8440286687daefb6c98f4, user.id=e3b38493ba904759badc800e396befda) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:07:21.325568 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2e29b2-9ad2-47f3-bb21-e4f5a957011c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:21.326982 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2e29b2-9ad2-47f3-bb21-e4f5a957011c None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:21.333882 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 438/873] 10.222.0.44 () {56 vars in 1374 bytes} [Wed Jun 29 11:07:21 2022] GET /identity/v3/role_assignments?user.id=e3b38493ba904759badc800e396befda&scope.project.id=ebe5dffa4bb8440286687daefb6c98f4&role.id=e95c2ce63e114cc19a9dda6ad21e794f => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:07:21.738167 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa8b609-191b-4e0a-86b0-e624dea3767a None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:21.738954 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa8b609-191b-4e0a-86b0-e624dea3767a None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:21.739448 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa8b609-191b-4e0a-86b0-e624dea3767a None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:21.741304 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 436/874] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:07:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:07:21.746745 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93a149c-5d34-41ec-83a4-98438b9ed8e6 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:21.747248 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93a149c-5d34-41ec-83a4-98438b9ed8e6 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:21.747761 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93a149c-5d34-41ec-83a4-98438b9ed8e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:21.765358 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-a93a149c-5d34-41ec-83a4-98438b9ed8e6 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:07:21.766886 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-a93a149c-5d34-41ec-83a4-98438b9ed8e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:07:21.825895 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:21.848873 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 439/875] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:07:21 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 11:07:21.873044 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa8b609-191b-4e0a-86b0-e624dea3767a None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:21.992482 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa8b609-191b-4e0a-86b0-e624dea3767a None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:22.016377 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6791062d-97e3-4e4e-8a52-ad743108f943 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:22.017306 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6791062d-97e3-4e4e-8a52-ad743108f943 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:22.017768 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6791062d-97e3-4e4e-8a52-ad743108f943 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:22.020241 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 440/876] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:07:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:07:22.025843 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a967e7-801a-404b-a579-163209d744b3 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:22.026138 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a967e7-801a-404b-a579-163209d744b3 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:22.026423 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a967e7-801a-404b-a579-163209d744b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:22.033390 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7277124c-422d-4262-8d93-cdc924ccdca5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:22.035409 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7277124c-422d-4262-8d93-cdc924ccdca5 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:22.035941 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7277124c-422d-4262-8d93-cdc924ccdca5 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:22.036471 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7277124c-422d-4262-8d93-cdc924ccdca5 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:22.043730 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-f8a967e7-801a-404b-a579-163209d744b3 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:07:22.044947 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-f8a967e7-801a-404b-a579-163209d744b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:07:22.045360 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:22.050052 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7277124c-422d-4262-8d93-cdc924ccdca5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 11:07:22.058430 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-7277124c-422d-4262-8d93-cdc924ccdca5 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 29 11:07:22.058430 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:07:22.058430 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:07:22.058430 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:07:22.058430 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:07:22.058430 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:07:22.058430 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:07:22.058430 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:07:22.058430 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:07:22.058430 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:07:22.058430 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:07:22.058430 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:07:22.058430 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 11:07:22.058430 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 11:07:22.058430 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 11:07:22.058430 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 11:07:22.064732 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 11:07:22.064732 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 11:07:22.064732 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 11:07:22.064732 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 11:07:22.064732 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:07:22.064732 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:07:22.064732 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:07:22.064732 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:07:22.064732 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:07:22.064732 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:07:22.064732 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:07:22.064732 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jun 29 11:07:22.064732 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:07:22.064732 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:07:22.064732 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:07:22.064732 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:07:22.065593 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:07:22.065593 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:07:22.065593 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:07:22.065593 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:07:22.065593 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 11:07:22.065593 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 11:07:22.065593 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 11:07:22.065593 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 11:07:22.065593 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 11:07:22.065593 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 11:07:22.065593 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 29 11:07:22.065593 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jun 29 11:07:22.065593 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 437/877] 10.222.0.44 () {56 vars in 1120 bytes} [Wed Jun 29 11:07:21 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:07:22.066325 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7277124c-422d-4262-8d93-cdc924ccdca5 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:22.136199 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 441/878] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:07:22 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (3 switches on core 0) Jun 29 11:07:22.151824 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a967e7-801a-404b-a579-163209d744b3 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:22.184810 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7277124c-422d-4262-8d93-cdc924ccdca5 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:22.223279 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f017ba-4a96-4c6e-8cfc-f255f331a801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:22.223971 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f017ba-4a96-4c6e-8cfc-f255f331a801 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:22.224509 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f017ba-4a96-4c6e-8cfc-f255f331a801 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:22.224791 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f017ba-4a96-4c6e-8cfc-f255f331a801 None admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:22.226129 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f017ba-4a96-4c6e-8cfc-f255f331a801 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:07:22.226443 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f017ba-4a96-4c6e-8cfc-f255f331a801 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:22.228142 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f017ba-4a96-4c6e-8cfc-f255f331a801 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:22.248151 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 438/879] 10.222.0.44 () {56 vars in 1115 bytes} [Wed Jun 29 11:07:22 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:07:22.250651 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f017ba-4a96-4c6e-8cfc-f255f331a801 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:22.276980 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a967e7-801a-404b-a579-163209d744b3 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:22.289425 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:22.312991 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529df197-9f04-4783-a93b-f7bc3d6ca949 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:22.313793 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529df197-9f04-4783-a93b-f7bc3d6ca949 None admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:22.314144 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529df197-9f04-4783-a93b-f7bc3d6ca949 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:22.314493 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529df197-9f04-4783-a93b-f7bc3d6ca949 None admin] PATH_INFO: `/v3/services` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:22.315292 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529df197-9f04-4783-a93b-f7bc3d6ca949 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:22.319776 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529df197-9f04-4783-a93b-f7bc3d6ca949 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:22.333900 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:22.374509 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f017ba-4a96-4c6e-8cfc-f255f331a801 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:22.395048 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 442/880] 10.222.0.44 () {60 vars in 1141 bytes} [Wed Jun 29 11:07:22 2022] POST /identity/v3/services => generated 269 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:07:22.427992 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b862339-b4a4-4603-bc44-e5c10e2d8b19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:22.429181 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b862339-b4a4-4603-bc44-e5c10e2d8b19 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:22.429713 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b862339-b4a4-4603-bc44-e5c10e2d8b19 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:22.430252 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b862339-b4a4-4603-bc44-e5c10e2d8b19 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:22.446185 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b862339-b4a4-4603-bc44-e5c10e2d8b19 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:22.448426 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b862339-b4a4-4603-bc44-e5c10e2d8b19 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:22.470702 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-8b862339-b4a4-4603-bc44-e5c10e2d8b19 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 11:07:22.470702 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:07:22.470702 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:07:22.470702 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:07:22.470702 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:07:22.470702 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:07:22.470702 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:07:22.470702 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:07:22.470702 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:07:22.470702 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:07:22.470702 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:07:22.470702 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:07:22.470702 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 11:07:22.470702 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 11:07:22.470702 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 11:07:22.471676 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 11:07:22.471676 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:07:22.471676 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:07:22.471676 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:07:22.471676 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:07:22.471676 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:07:22.471676 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:07:22.471676 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:07:22.471676 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jun 29 11:07:22.471676 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:07:22.471676 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:07:22.471676 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:07:22.471676 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:07:22.471676 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:07:22.471676 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:07:22.471676 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:07:22.471676 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:07:22.472651 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 11:07:22.472651 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 11:07:22.472651 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 11:07:22.472651 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jun 29 11:07:22.475107 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 439/881] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jun 29 11:07:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 11:07:22.480914 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529df197-9f04-4783-a93b-f7bc3d6ca949 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:22.598584 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529df197-9f04-4783-a93b-f7bc3d6ca949 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:22.634910 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff8d12f-990d-4494-860a-066f7e21535b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:22.636032 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff8d12f-990d-4494-860a-066f7e21535b None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:22.636519 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff8d12f-990d-4494-860a-066f7e21535b None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:22.636992 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff8d12f-990d-4494-860a-066f7e21535b None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:22.637908 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff8d12f-990d-4494-860a-066f7e21535b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:07:22.638437 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff8d12f-990d-4494-860a-066f7e21535b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:22.640022 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff8d12f-990d-4494-860a-066f7e21535b None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:22.652843 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 443/882] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Jun 29 11:07:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 11:07:22.661328 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b862339-b4a4-4603-bc44-e5c10e2d8b19 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:22.770988 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b862339-b4a4-4603-bc44-e5c10e2d8b19 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:22.780782 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:22.806330 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee8121d-48ff-4d41-9c36-6136bd8df15f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:22.807149 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee8121d-48ff-4d41-9c36-6136bd8df15f None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:22.813368 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee8121d-48ff-4d41-9c36-6136bd8df15f None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:22.813368 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee8121d-48ff-4d41-9c36-6136bd8df15f None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:22.820940 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee8121d-48ff-4d41-9c36-6136bd8df15f None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:22.820940 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee8121d-48ff-4d41-9c36-6136bd8df15f None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:22.837428 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-9ee8121d-48ff-4d41-9c36-6136bd8df15f None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 29 11:07:22.837428 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:07:22.837428 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:07:22.837428 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:07:22.837428 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:07:22.837428 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:07:22.837428 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:07:22.837428 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:07:22.837428 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:07:22.837428 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:07:22.837428 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:07:22.837428 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:07:22.837428 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 11:07:22.837428 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 11:07:22.837428 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 11:07:22.837428 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 11:07:22.838399 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:07:22.838399 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:07:22.838399 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 11:07:22.838399 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 11:07:22.838399 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 11:07:22.838399 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 11:07:22.838399 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:07:22.838399 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:07:22.838399 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:07:22.838399 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:07:22.838399 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:07:22.838399 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jun 29 11:07:22.838399 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:07:22.838399 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:07:22.838399 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:07:22.838399 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:07:22.838399 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:07:22.838399 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:07:22.839596 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:07:22.839596 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:07:22.839596 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 11:07:22.839596 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 11:07:22.839596 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 11:07:22.839596 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 11:07:22.839596 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 11:07:22.839596 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 11:07:22.839596 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 29 11:07:22.839596 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jun 29 11:07:22.839596 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 440/883] 10.222.0.44 () {56 vars in 1117 bytes} [Wed Jun 29 11:07:22 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 11:07:22.848992 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff8d12f-990d-4494-860a-066f7e21535b None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:22.868245 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f333a1e-7784-49a0-b118-07165d63b8dd None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:22.868543 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f333a1e-7784-49a0-b118-07165d63b8dd None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:22.868797 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f333a1e-7784-49a0-b118-07165d63b8dd None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:22.870291 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 441/884] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:07:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 29 11:07:22.874977 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd62274-38d8-4270-a5a5-bf30b10d919f None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:22.875278 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd62274-38d8-4270-a5a5-bf30b10d919f None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:22.875530 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd62274-38d8-4270-a5a5-bf30b10d919f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:22.888230 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:22.895842 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-1cd62274-38d8-4270-a5a5-bf30b10d919f None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:07:22.897097 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-1cd62274-38d8-4270-a5a5-bf30b10d919f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:07:22.979523 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff8d12f-990d-4494-860a-066f7e21535b None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:22.992255 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 442/885] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:07:22 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 29 11:07:23.014809 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd62274-38d8-4270-a5a5-bf30b10d919f None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:23.023134 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be0b984-402d-41aa-ba96-2373eda526b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:23.023816 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be0b984-402d-41aa-ba96-2373eda526b1 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:23.024129 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be0b984-402d-41aa-ba96-2373eda526b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:23.024404 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be0b984-402d-41aa-ba96-2373eda526b1 None admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:23.025706 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be0b984-402d-41aa-ba96-2373eda526b1 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:07:23.026022 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be0b984-402d-41aa-ba96-2373eda526b1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:23.028257 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be0b984-402d-41aa-ba96-2373eda526b1 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:23.053240 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 444/886] 10.222.0.44 () {56 vars in 1149 bytes} [Wed Jun 29 11:07:22 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:07:23.061064 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be0b984-402d-41aa-ba96-2373eda526b1 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:23.135953 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:23.140982 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd62274-38d8-4270-a5a5-bf30b10d919f None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:23.180848 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:23.184415 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c1aff5-ea29-4759-9cd7-a261b2ddddf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:23.185419 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c1aff5-ea29-4759-9cd7-a261b2ddddf7 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:23.185849 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c1aff5-ea29-4759-9cd7-a261b2ddddf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:23.186280 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c1aff5-ea29-4759-9cd7-a261b2ddddf7 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:23.187079 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c1aff5-ea29-4759-9cd7-a261b2ddddf7 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:23.188896 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c1aff5-ea29-4759-9cd7-a261b2ddddf7 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:23.195738 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be0b984-402d-41aa-ba96-2373eda526b1 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:23.202747 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-28c1aff5-ea29-4759-9cd7-a261b2ddddf7 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 29 11:07:23.202747 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:07:23.202747 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 29 11:07:23.202747 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 29 11:07:23.202747 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 29 11:07:23.202747 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 29 11:07:23.202747 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 29 11:07:23.202747 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 11:07:23.202747 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 29 11:07:23.202747 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jun 29 11:07:23.202747 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 29 11:07:23.202747 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jun 29 11:07:23.202747 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:07:23.202747 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:07:23.202747 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:07:23.202747 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:07:23.202747 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:07:23.203947 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:07:23.203947 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:07:23.203947 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:07:23.203947 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:07:23.203947 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:07:23.203947 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:07:23.203947 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 29 11:07:23.203947 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 29 11:07:23.203947 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 29 11:07:23.203947 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 29 11:07:23.203947 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:07:23.203947 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:07:23.203947 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:07:23.203947 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:07:23.203947 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:07:23.203947 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:07:23.204802 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:07:23.204802 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jun 29 11:07:23.204802 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:07:23.204802 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:07:23.204802 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:07:23.204802 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:07:23.204802 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:07:23.204802 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:07:23.204802 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:07:23.204802 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:07:23.204802 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 29 11:07:23.204802 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 11:07:23.204802 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 29 11:07:23.204802 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jun 29 11:07:23.208243 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 443/887] 10.222.0.44 () {56 vars in 1114 bytes} [Wed Jun 29 11:07:23 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 11:07:23.215407 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c1aff5-ea29-4759-9cd7-a261b2ddddf7 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:23.238194 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adb6691-942a-4e6b-a314-928d00a12ef6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:23.238848 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adb6691-942a-4e6b-a314-928d00a12ef6 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:23.239142 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adb6691-942a-4e6b-a314-928d00a12ef6 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:23.239421 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adb6691-942a-4e6b-a314-928d00a12ef6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:23.314922 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adb6691-942a-4e6b-a314-928d00a12ef6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:23.316775 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adb6691-942a-4e6b-a314-928d00a12ef6 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:23.350911 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c1aff5-ea29-4759-9cd7-a261b2ddddf7 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:23.399659 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31d536a-8d67-48b9-91d3-ae8072f0a7a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:23.400696 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31d536a-8d67-48b9-91d3-ae8072f0a7a9 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:23.401717 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31d536a-8d67-48b9-91d3-ae8072f0a7a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:23.402238 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31d536a-8d67-48b9-91d3-ae8072f0a7a9 None admin] PATH_INFO: `/v3/services` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:23.403116 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31d536a-8d67-48b9-91d3-ae8072f0a7a9 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:07:23.403943 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31d536a-8d67-48b9-91d3-ae8072f0a7a9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:23.405395 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31d536a-8d67-48b9-91d3-ae8072f0a7a9 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:23.417750 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 444/888] 10.222.0.44 () {56 vars in 1114 bytes} [Wed Jun 29 11:07:23 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:07:23.418772 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31d536a-8d67-48b9-91d3-ae8072f0a7a9 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:23.426042 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:23.445433 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 445/889] 10.222.0.44 () {58 vars in 1282 bytes} [Wed Jun 29 11:07:23 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 29 11:07:23.451873 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adb6691-942a-4e6b-a314-928d00a12ef6 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:23.455120 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:23.538490 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31d536a-8d67-48b9-91d3-ae8072f0a7a9 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:23.569214 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adb6691-942a-4e6b-a314-928d00a12ef6 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:23.574314 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6220999-0dc9-4792-9237-f53e4a03f3d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:23.574986 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6220999-0dc9-4792-9237-f53e4a03f3d7 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:23.575269 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6220999-0dc9-4792-9237-f53e4a03f3d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:23.575544 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6220999-0dc9-4792-9237-f53e4a03f3d7 None admin] PATH_INFO: `/v3/services` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:23.576285 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6220999-0dc9-4792-9237-f53e4a03f3d7 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:07:23.576593 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6220999-0dc9-4792-9237-f53e4a03f3d7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:23.577965 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6220999-0dc9-4792-9237-f53e4a03f3d7 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:23.583901 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 445/890] 10.222.0.44 () {56 vars in 1114 bytes} [Wed Jun 29 11:07:23 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:07:23.584793 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40cab35-108e-47fa-a021-52a3ffd332bc None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:23.585077 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40cab35-108e-47fa-a021-52a3ffd332bc None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:23.585353 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40cab35-108e-47fa-a021-52a3ffd332bc None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:23.586750 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 446/891] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:07:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:07:23.588836 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40cab35-108e-47fa-a021-52a3ffd332bc None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:23.614971 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d1f125-2ba0-4748-9878-7d4cedaa9072 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:23.616101 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d1f125-2ba0-4748-9878-7d4cedaa9072 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:23.616575 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d1f125-2ba0-4748-9878-7d4cedaa9072 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:23.616855 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d1f125-2ba0-4748-9878-7d4cedaa9072 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:23.627437 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d1f125-2ba0-4748-9878-7d4cedaa9072 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:23.629465 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d1f125-2ba0-4748-9878-7d4cedaa9072 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:23.647382 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-98d1f125-2ba0-4748-9878-7d4cedaa9072 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 11:07:23.647382 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:07:23.647382 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:07:23.647382 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:07:23.647382 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:07:23.647382 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:07:23.647382 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:07:23.647382 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:07:23.647382 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:07:23.647382 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:07:23.647382 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:07:23.647382 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:07:23.647382 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 11:07:23.647382 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 11:07:23.647382 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 11:07:23.648359 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 11:07:23.648359 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:07:23.648359 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:07:23.648359 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:07:23.648359 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:07:23.648359 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:07:23.648359 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:07:23.648359 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:07:23.648359 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jun 29 11:07:23.648359 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:07:23.648359 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:07:23.648359 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:07:23.648359 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:07:23.648359 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:07:23.648359 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:07:23.648359 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:07:23.648359 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:07:23.649257 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 11:07:23.649257 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 11:07:23.649257 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 11:07:23.649257 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:07:23.649257 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 446/892] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jun 29 11:07:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 11:07:23.650134 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaf3260-84b3-4dd1-a601-aa7dc3a83186 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:23.650370 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaf3260-84b3-4dd1-a601-aa7dc3a83186 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:23.650643 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaf3260-84b3-4dd1-a601-aa7dc3a83186 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:23.666561 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-2eaf3260-84b3-4dd1-a601-aa7dc3a83186 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:07:23.667784 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-2eaf3260-84b3-4dd1-a601-aa7dc3a83186 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:07:23.695135 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:23.703450 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:23.706858 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40cab35-108e-47fa-a021-52a3ffd332bc None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:23.736822 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 447/893] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:07:23 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 11:07:23.743625 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f63025-10f2-48c5-92b7-911c84c936a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:23.744311 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f63025-10f2-48c5-92b7-911c84c936a5 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:23.744597 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f63025-10f2-48c5-92b7-911c84c936a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:23.744877 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f63025-10f2-48c5-92b7-911c84c936a5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:23.745208 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaf3260-84b3-4dd1-a601-aa7dc3a83186 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:23.745698 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f63025-10f2-48c5-92b7-911c84c936a5 None admin] RBAC: Adding query filter params (interface=public, service_id=d0f5d026111743cb91a17280b83063c3, region_id=RegionOne) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:07:23.745962 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f63025-10f2-48c5-92b7-911c84c936a5 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:23.747319 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f63025-10f2-48c5-92b7-911c84c936a5 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:23.760225 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 447/894] 10.222.0.44 () {56 vars in 1251 bytes} [Wed Jun 29 11:07:23 2022] GET /identity/v3/endpoints?service_id=d0f5d026111743cb91a17280b83063c3&interface=public®ion_id=RegionOne => generated 194 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 11:07:23.762729 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f63025-10f2-48c5-92b7-911c84c936a5 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:23.870023 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaf3260-84b3-4dd1-a601-aa7dc3a83186 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:23.874910 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f63025-10f2-48c5-92b7-911c84c936a5 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:23.904475 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eeb1643-bf02-47ee-8b8b-11a95e8f7986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:23.905076 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eeb1643-bf02-47ee-8b8b-11a95e8f7986 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:23.905360 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eeb1643-bf02-47ee-8b8b-11a95e8f7986 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:23.905637 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eeb1643-bf02-47ee-8b8b-11a95e8f7986 None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:23.906355 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eeb1643-bf02-47ee-8b8b-11a95e8f7986 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:07:23.906663 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eeb1643-bf02-47ee-8b8b-11a95e8f7986 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:23.908066 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eeb1643-bf02-47ee-8b8b-11a95e8f7986 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:23.910356 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f34d176-d465-44bc-9cc4-07ec556c9998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:23.911018 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f34d176-d465-44bc-9cc4-07ec556c9998 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:23.911308 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f34d176-d465-44bc-9cc4-07ec556c9998 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:23.911593 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f34d176-d465-44bc-9cc4-07ec556c9998 None admin] PATH_INFO: `/v3/services` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:23.912308 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f34d176-d465-44bc-9cc4-07ec556c9998 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:23.913675 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f34d176-d465-44bc-9cc4-07ec556c9998 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:23.924606 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 448/895] 10.222.0.44 () {56 vars in 1087 bytes} [Wed Jun 29 11:07:23 2022] GET /identity/v3/services => generated 1268 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 11:07:23.926232 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f34d176-d465-44bc-9cc4-07ec556c9998 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:23.930792 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 448/896] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Jun 29 11:07:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:07:23.931779 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eeb1643-bf02-47ee-8b8b-11a95e8f7986 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:23.960492 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:23.967538 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:24.059502 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f34d176-d465-44bc-9cc4-07ec556c9998 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:24.074551 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eeb1643-bf02-47ee-8b8b-11a95e8f7986 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:24.106222 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541aa5c8-27ba-4b64-8a86-a302f46dbcdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:24.106945 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541aa5c8-27ba-4b64-8a86-a302f46dbcdc None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:24.107281 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541aa5c8-27ba-4b64-8a86-a302f46dbcdc None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:24.107701 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541aa5c8-27ba-4b64-8a86-a302f46dbcdc None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:24.108981 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541aa5c8-27ba-4b64-8a86-a302f46dbcdc None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:24.110373 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541aa5c8-27ba-4b64-8a86-a302f46dbcdc None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:24.120586 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b30b1b-f599-4c64-8792-c6cd0d140410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:24.122348 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b30b1b-f599-4c64-8792-c6cd0d140410 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:24.123103 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b30b1b-f599-4c64-8792-c6cd0d140410 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:24.123850 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b30b1b-f599-4c64-8792-c6cd0d140410 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:24.125111 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-541aa5c8-27ba-4b64-8a86-a302f46dbcdc None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 29 11:07:24.125111 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:07:24.125111 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 29 11:07:24.125111 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 29 11:07:24.125111 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 29 11:07:24.125111 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 29 11:07:24.125111 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 29 11:07:24.125111 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 11:07:24.125111 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 29 11:07:24.125111 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jun 29 11:07:24.125111 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 29 11:07:24.125111 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jun 29 11:07:24.125111 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:07:24.125111 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:07:24.125111 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:07:24.125111 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:07:24.125111 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:07:24.126437 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:07:24.126437 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:07:24.126437 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:07:24.126437 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:07:24.126437 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:07:24.126437 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:07:24.126437 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 29 11:07:24.126437 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 29 11:07:24.126437 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 29 11:07:24.126437 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 29 11:07:24.126437 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:07:24.126437 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:07:24.126437 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:07:24.126437 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:07:24.126437 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:07:24.126437 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:07:24.127353 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:07:24.127353 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jun 29 11:07:24.127353 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:07:24.127353 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:07:24.127353 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:07:24.127353 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:07:24.127353 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:07:24.127353 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:07:24.127353 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:07:24.127353 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:07:24.127353 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 29 11:07:24.127353 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 11:07:24.127353 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 29 11:07:24.127353 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jun 29 11:07:24.131915 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 449/897] 10.222.0.44 () {56 vars in 1126 bytes} [Wed Jun 29 11:07:23 2022] GET /identity/v3/services/object-store => generated 93 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:07:24.134833 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b30b1b-f599-4c64-8792-c6cd0d140410 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:24.136337 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b30b1b-f599-4c64-8792-c6cd0d140410 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:24.136955 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541aa5c8-27ba-4b64-8a86-a302f46dbcdc None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:24.155930 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-71b30b1b-f599-4c64-8792-c6cd0d140410 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 11:07:24.155930 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:07:24.155930 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:07:24.155930 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:07:24.155930 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:07:24.155930 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:07:24.155930 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:07:24.155930 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:07:24.155930 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:07:24.155930 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:07:24.155930 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:07:24.155930 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:07:24.155930 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 11:07:24.155930 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 11:07:24.155930 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 11:07:24.156850 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 11:07:24.156850 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:07:24.156850 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:07:24.156850 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:07:24.156850 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:07:24.156850 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:07:24.156850 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:07:24.156850 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:07:24.156850 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jun 29 11:07:24.156850 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:07:24.156850 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:07:24.156850 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:07:24.156850 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:07:24.156850 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:07:24.156850 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:07:24.156850 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:07:24.156850 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:07:24.157755 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 11:07:24.157755 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 11:07:24.157755 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 11:07:24.157755 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 11:07:24.157755 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 11:07:24.157755 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 11:07:24.157755 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 11:07:24.157755 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:07:24.157755 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 449/898] 10.222.0.44 () {56 vars in 1111 bytes} [Wed Jun 29 11:07:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 11:07:24.164839 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b30b1b-f599-4c64-8792-c6cd0d140410 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:24.252316 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:24.266424 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:24.267530 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541aa5c8-27ba-4b64-8a86-a302f46dbcdc None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:24.281773 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b30b1b-f599-4c64-8792-c6cd0d140410 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:24.309996 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e907d4-5442-46f3-8f22-b21ab688186d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:24.311073 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e907d4-5442-46f3-8f22-b21ab688186d None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:24.311415 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e907d4-5442-46f3-8f22-b21ab688186d None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:24.311750 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e907d4-5442-46f3-8f22-b21ab688186d None admin] PATH_INFO: `/v3/services` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:24.313173 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e907d4-5442-46f3-8f22-b21ab688186d None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:07:24.314124 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e907d4-5442-46f3-8f22-b21ab688186d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:24.315666 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e907d4-5442-46f3-8f22-b21ab688186d None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:24.323147 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 450/899] 10.222.0.44 () {56 vars in 1122 bytes} [Wed Jun 29 11:07:24 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 11:07:24.335882 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e19081b-6305-464f-9ddf-ad35aa2544d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:24.335882 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e19081b-6305-464f-9ddf-ad35aa2544d9 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:24.335882 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e19081b-6305-464f-9ddf-ad35aa2544d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:24.335882 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e19081b-6305-464f-9ddf-ad35aa2544d9 None admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:24.336292 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e19081b-6305-464f-9ddf-ad35aa2544d9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:07:24.336292 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e19081b-6305-464f-9ddf-ad35aa2544d9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:24.336418 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e907d4-5442-46f3-8f22-b21ab688186d None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:24.336500 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e19081b-6305-464f-9ddf-ad35aa2544d9 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:24.344672 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 450/900] 10.222.0.44 () {56 vars in 1148 bytes} [Wed Jun 29 11:07:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:07:24.350778 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e19081b-6305-464f-9ddf-ad35aa2544d9 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:24.462063 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e907d4-5442-46f3-8f22-b21ab688186d None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:24.487245 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e19081b-6305-464f-9ddf-ad35aa2544d9 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:24.499994 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c5128a-6147-4f78-b99a-e32292cbde7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:24.501082 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c5128a-6147-4f78-b99a-e32292cbde7c None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:24.501571 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c5128a-6147-4f78-b99a-e32292cbde7c None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:24.502069 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c5128a-6147-4f78-b99a-e32292cbde7c None admin] PATH_INFO: `/v3/services` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:24.503078 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c5128a-6147-4f78-b99a-e32292cbde7c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:07:24.504229 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c5128a-6147-4f78-b99a-e32292cbde7c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:24.505891 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c5128a-6147-4f78-b99a-e32292cbde7c None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:24.511551 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 451/901] 10.222.0.44 () {56 vars in 1122 bytes} [Wed Jun 29 11:07:24 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 11:07:24.526295 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d927441b-90f5-476d-bcd3-1f00fbd246b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:24.526977 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d927441b-90f5-476d-bcd3-1f00fbd246b2 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:24.527264 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d927441b-90f5-476d-bcd3-1f00fbd246b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:24.527543 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d927441b-90f5-476d-bcd3-1f00fbd246b2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:24.528439 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d927441b-90f5-476d-bcd3-1f00fbd246b2 None admin] RBAC: Adding query filter params (role.id=3e8ef91b147f4b6b803ef07f3301be3f, scope.project.id=ebe5dffa4bb8440286687daefb6c98f4, user.id=55487c2107a44c15b656b4865ee0d4fa) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:07:24.528818 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d927441b-90f5-476d-bcd3-1f00fbd246b2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:24.530451 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d927441b-90f5-476d-bcd3-1f00fbd246b2 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:24.537679 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 451/902] 10.222.0.44 () {56 vars in 1374 bytes} [Wed Jun 29 11:07:24 2022] GET /identity/v3/role_assignments?user.id=55487c2107a44c15b656b4865ee0d4fa&scope.project.id=ebe5dffa4bb8440286687daefb6c98f4&role.id=3e8ef91b147f4b6b803ef07f3301be3f => generated 259 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:07:24.555398 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cecebe6-15bf-444f-bd51-615317409688 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:24.555956 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cecebe6-15bf-444f-bd51-615317409688 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:24.556390 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cecebe6-15bf-444f-bd51-615317409688 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:24.561868 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 452/903] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:07:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:07:24.563764 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f7d060-ed93-4faf-849e-b019cbf05230 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:24.564100 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f7d060-ed93-4faf-849e-b019cbf05230 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:24.564377 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f7d060-ed93-4faf-849e-b019cbf05230 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:24.587653 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-66f7d060-ed93-4faf-849e-b019cbf05230 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:07:24.588930 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-66f7d060-ed93-4faf-849e-b019cbf05230 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:07:24.601373 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:24.671276 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 452/904] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:07:24 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 29 11:07:24.695400 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cecebe6-15bf-444f-bd51-615317409688 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:24.715968 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:24.817346 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cecebe6-15bf-444f-bd51-615317409688 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:24.851890 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de32e8f0-defc-4786-a1fe-174286af9e05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:24.853016 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de32e8f0-defc-4786-a1fe-174286af9e05 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:24.853444 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de32e8f0-defc-4786-a1fe-174286af9e05 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:24.853856 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de32e8f0-defc-4786-a1fe-174286af9e05 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:24.854653 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de32e8f0-defc-4786-a1fe-174286af9e05 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:24.856179 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de32e8f0-defc-4786-a1fe-174286af9e05 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:24.869006 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-de32e8f0-defc-4786-a1fe-174286af9e05 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 29 11:07:24.869006 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:07:24.869006 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 29 11:07:24.869006 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 29 11:07:24.869006 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 29 11:07:24.869006 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 29 11:07:24.869006 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 29 11:07:24.869006 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 11:07:24.869006 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 29 11:07:24.869006 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jun 29 11:07:24.869006 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 29 11:07:24.869006 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jun 29 11:07:24.869006 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:07:24.869006 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:07:24.869006 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:07:24.869006 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:07:24.870180 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:07:24.870180 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:07:24.870180 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:07:24.870180 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:07:24.870180 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:07:24.870180 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:07:24.870180 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:07:24.870180 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 29 11:07:24.870180 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 29 11:07:24.870180 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 29 11:07:24.870180 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 29 11:07:24.870180 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:07:24.870180 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:07:24.870180 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:07:24.870180 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:07:24.870180 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:07:24.871224 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:07:24.871224 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:07:24.871224 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jun 29 11:07:24.871224 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:07:24.871224 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:07:24.871224 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:07:24.871224 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:07:24.871224 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:07:24.871224 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:07:24.871224 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:07:24.871224 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:07:24.871224 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 29 11:07:24.871224 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 11:07:24.871224 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 29 11:07:24.871224 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jun 29 11:07:24.873829 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 453/905] 10.222.0.44 () {56 vars in 1132 bytes} [Wed Jun 29 11:07:24 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:07:24.880476 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f7d060-ed93-4faf-849e-b019cbf05230 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:24.990226 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f7d060-ed93-4faf-849e-b019cbf05230 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:25.025683 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc76ba1-e069-453f-b5ca-cf708ad7e5d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:25.026625 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc76ba1-e069-453f-b5ca-cf708ad7e5d0 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:25.027080 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc76ba1-e069-453f-b5ca-cf708ad7e5d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:25.027540 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc76ba1-e069-453f-b5ca-cf708ad7e5d0 None admin] PATH_INFO: `/v3/services` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:25.028469 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc76ba1-e069-453f-b5ca-cf708ad7e5d0 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:07:25.028969 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc76ba1-e069-453f-b5ca-cf708ad7e5d0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:25.030469 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc76ba1-e069-453f-b5ca-cf708ad7e5d0 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:25.036063 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 453/906] 10.222.0.44 () {56 vars in 1126 bytes} [Wed Jun 29 11:07:24 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:07:25.043871 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de32e8f0-defc-4786-a1fe-174286af9e05 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:25.122129 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:25.148493 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de32e8f0-defc-4786-a1fe-174286af9e05 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:25.185111 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e26808-06f5-4387-81fd-44c857ea665b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:25.185783 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e26808-06f5-4387-81fd-44c857ea665b None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:25.186070 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e26808-06f5-4387-81fd-44c857ea665b None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:25.186346 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e26808-06f5-4387-81fd-44c857ea665b None admin] PATH_INFO: `/v3/services` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:25.187055 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e26808-06f5-4387-81fd-44c857ea665b None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:07:25.187363 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e26808-06f5-4387-81fd-44c857ea665b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:25.188759 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e26808-06f5-4387-81fd-44c857ea665b None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:25.194721 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 454/907] 10.222.0.44 () {56 vars in 1126 bytes} [Wed Jun 29 11:07:25 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:07:25.200432 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc76ba1-e069-453f-b5ca-cf708ad7e5d0 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:25.213327 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:25.304860 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc76ba1-e069-453f-b5ca-cf708ad7e5d0 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:25.339950 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e635b69-fcb8-4031-8153-735d07226318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:25.340614 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e635b69-fcb8-4031-8153-735d07226318 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:25.340896 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e635b69-fcb8-4031-8153-735d07226318 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:25.341172 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e635b69-fcb8-4031-8153-735d07226318 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:25.341951 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e635b69-fcb8-4031-8153-735d07226318 None admin] RBAC: Adding query filter params (interface=public, service_id=4840a4271cbb463a9b5bbbc97c64705a, region_id=RegionOne) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:07:25.342257 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e635b69-fcb8-4031-8153-735d07226318 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:25.343607 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e635b69-fcb8-4031-8153-735d07226318 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:25.352606 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 454/908] 10.222.0.44 () {56 vars in 1251 bytes} [Wed Jun 29 11:07:25 2022] GET /identity/v3/endpoints?service_id=4840a4271cbb463a9b5bbbc97c64705a&interface=public®ion_id=RegionOne => generated 194 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:07:25.358129 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e26808-06f5-4387-81fd-44c857ea665b None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:25.462715 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e26808-06f5-4387-81fd-44c857ea665b None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:25.497151 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eee0a7e-020e-47a5-a716-919eb8688968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:25.498055 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eee0a7e-020e-47a5-a716-919eb8688968 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:25.498509 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eee0a7e-020e-47a5-a716-919eb8688968 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:25.498951 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eee0a7e-020e-47a5-a716-919eb8688968 None admin] PATH_INFO: `/v3/services` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:25.499790 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eee0a7e-020e-47a5-a716-919eb8688968 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:25.501333 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eee0a7e-020e-47a5-a716-919eb8688968 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:25.507668 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 455/909] 10.222.0.44 () {56 vars in 1087 bytes} [Wed Jun 29 11:07:25 2022] GET /identity/v3/services => generated 1268 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 11:07:26.134947 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ff1036-f3ed-4dbf-9675-64abfe0a60a6 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:26.135339 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ff1036-f3ed-4dbf-9675-64abfe0a60a6 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:26.135521 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ff1036-f3ed-4dbf-9675-64abfe0a60a6 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:26.137141 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 455/910] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:07:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:07:26.142302 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136d61de-6edb-4a14-b13d-c3ed23512e92 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:26.142597 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136d61de-6edb-4a14-b13d-c3ed23512e92 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:26.142873 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136d61de-6edb-4a14-b13d-c3ed23512e92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:26.148229 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:26.161804 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-136d61de-6edb-4a14-b13d-c3ed23512e92 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:07:26.163715 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-136d61de-6edb-4a14-b13d-c3ed23512e92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:07:26.234947 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 456/911] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:07:26 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 11:07:26.258758 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ff1036-f3ed-4dbf-9675-64abfe0a60a6 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:26.338092 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:26.369418 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ff1036-f3ed-4dbf-9675-64abfe0a60a6 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:26.409991 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd0ace2-35e0-4f23-a47f-dcd3bc395763 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:26.411159 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd0ace2-35e0-4f23-a47f-dcd3bc395763 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:26.411639 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd0ace2-35e0-4f23-a47f-dcd3bc395763 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:26.412135 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd0ace2-35e0-4f23-a47f-dcd3bc395763 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:26.413112 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd0ace2-35e0-4f23-a47f-dcd3bc395763 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:26.414747 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd0ace2-35e0-4f23-a47f-dcd3bc395763 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:26.427033 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-bbd0ace2-35e0-4f23-a47f-dcd3bc395763 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 29 11:07:26.427033 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:07:26.427033 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 29 11:07:26.427033 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 29 11:07:26.427033 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 29 11:07:26.427033 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 29 11:07:26.427033 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 29 11:07:26.427033 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 11:07:26.427033 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 29 11:07:26.427033 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:07:26.427033 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 29 11:07:26.427033 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:07:26.427033 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:07:26.427033 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:07:26.427033 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:07:26.427033 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:07:26.427033 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:07:26.428002 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:07:26.428002 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:07:26.428002 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:07:26.428002 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:07:26.428002 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:07:26.428002 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:07:26.428002 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 29 11:07:26.428002 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 29 11:07:26.428002 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 29 11:07:26.428002 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 29 11:07:26.428002 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:07:26.428002 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:07:26.428002 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:07:26.428002 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:07:26.428002 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:07:26.428002 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:07:26.428853 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:07:26.428853 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jun 29 11:07:26.428853 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:07:26.428853 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:07:26.428853 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:07:26.428853 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:07:26.428853 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:07:26.428853 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:07:26.428853 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:07:26.428853 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:07:26.428853 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 29 11:07:26.428853 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 11:07:26.428853 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 29 11:07:26.428853 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:07:26.431754 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 456/912] 10.222.0.44 () {56 vars in 1114 bytes} [Wed Jun 29 11:07:26 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:07:26.437908 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136d61de-6edb-4a14-b13d-c3ed23512e92 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:26.544614 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136d61de-6edb-4a14-b13d-c3ed23512e92 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:26.577212 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:26.580778 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98b25e3-dde9-4b59-a8c8-1d14fcbc2bd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:26.581546 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98b25e3-dde9-4b59-a8c8-1d14fcbc2bd8 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:26.581840 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98b25e3-dde9-4b59-a8c8-1d14fcbc2bd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:26.582171 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98b25e3-dde9-4b59-a8c8-1d14fcbc2bd8 None admin] PATH_INFO: `/v3/services` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:26.582883 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98b25e3-dde9-4b59-a8c8-1d14fcbc2bd8 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:07:26.583194 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98b25e3-dde9-4b59-a8c8-1d14fcbc2bd8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:26.584587 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98b25e3-dde9-4b59-a8c8-1d14fcbc2bd8 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:26.629815 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 457/913] 10.222.0.44 () {56 vars in 1114 bytes} [Wed Jun 29 11:07:26 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:07:26.629950 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd0ace2-35e0-4f23-a47f-dcd3bc395763 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:26.630033 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08c0e61-15ab-4762-a1b6-c179cf32a021 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:26.630033 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08c0e61-15ab-4762-a1b6-c179cf32a021 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:26.630033 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08c0e61-15ab-4762-a1b6-c179cf32a021 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:26.630033 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 458/914] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:07:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 29 11:07:26.630033 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8abd97-6841-44d0-9e03-fc4b47231c0d None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:26.630033 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8abd97-6841-44d0-9e03-fc4b47231c0d None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:26.630394 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8abd97-6841-44d0-9e03-fc4b47231c0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:26.630394 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-0b8abd97-6841-44d0-9e03-fc4b47231c0d None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:07:26.630394 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-0b8abd97-6841-44d0-9e03-fc4b47231c0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:07:26.708076 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 459/915] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:07:26 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 11:07:26.724661 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd0ace2-35e0-4f23-a47f-dcd3bc395763 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:26.729810 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8abd97-6841-44d0-9e03-fc4b47231c0d None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:26.766173 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf7f378-ef2c-40d6-864a-31008744d637 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:26.766842 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf7f378-ef2c-40d6-864a-31008744d637 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:26.767126 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf7f378-ef2c-40d6-864a-31008744d637 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:26.767403 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf7f378-ef2c-40d6-864a-31008744d637 None admin] PATH_INFO: `/v3/services` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:26.768136 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf7f378-ef2c-40d6-864a-31008744d637 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:07:26.768440 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf7f378-ef2c-40d6-864a-31008744d637 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:26.769826 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf7f378-ef2c-40d6-864a-31008744d637 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:26.771819 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:26.774984 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 457/916] 10.222.0.44 () {56 vars in 1114 bytes} [Wed Jun 29 11:07:26 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:07:26.776219 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c080b63-37d8-4ca5-bc9e-95456df48a8d None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:26.776505 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c080b63-37d8-4ca5-bc9e-95456df48a8d None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:26.776782 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c080b63-37d8-4ca5-bc9e-95456df48a8d None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:26.778180 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 458/917] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:07:26 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:07:26.782704 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c080b63-37d8-4ca5-bc9e-95456df48a8d None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:26.843004 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8abd97-6841-44d0-9e03-fc4b47231c0d None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:26.869696 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:26.878155 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b674eac-b1e0-419c-86bb-7b3e936c48a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:26.881305 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b674eac-b1e0-419c-86bb-7b3e936c48a4 None admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:26.881305 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b674eac-b1e0-419c-86bb-7b3e936c48a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:26.881305 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b674eac-b1e0-419c-86bb-7b3e936c48a4 None admin] PATH_INFO: `/v3/services` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:26.881305 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b674eac-b1e0-419c-86bb-7b3e936c48a4 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:26.881602 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b674eac-b1e0-419c-86bb-7b3e936c48a4 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:26.889008 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c080b63-37d8-4ca5-bc9e-95456df48a8d None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:26.931775 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4743100-82f0-4999-93dc-92d0e691a10d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:26.933025 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4743100-82f0-4999-93dc-92d0e691a10d None admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:26.933606 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4743100-82f0-4999-93dc-92d0e691a10d None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:26.934114 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4743100-82f0-4999-93dc-92d0e691a10d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:26.935147 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4743100-82f0-4999-93dc-92d0e691a10d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:26.936967 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4743100-82f0-4999-93dc-92d0e691a10d None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:26.958464 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 460/918] 10.222.0.44 () {60 vars in 1141 bytes} [Wed Jun 29 11:07:26 2022] POST /identity/v3/services => generated 241 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 11:07:26.959891 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d0d0a3-f2d5-4479-a544-3491e30eaefa None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:26.960254 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d0d0a3-f2d5-4479-a544-3491e30eaefa None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:26.960602 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d0d0a3-f2d5-4479-a544-3491e30eaefa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:26.979203 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-f1d0d0a3-f2d5-4479-a544-3491e30eaefa None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:07:26.980635 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-f1d0d0a3-f2d5-4479-a544-3491e30eaefa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:07:27.025618 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 459/919] 10.222.0.44 () {60 vars in 1144 bytes} [Wed Jun 29 11:07:26 2022] POST /identity/v3/endpoints => generated 346 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:07:27.073597 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 461/920] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:07:26 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 11:07:27.108614 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4743100-82f0-4999-93dc-92d0e691a10d None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:27.208291 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:27.263510 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4743100-82f0-4999-93dc-92d0e691a10d None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:27.306584 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ee40ba-1174-475d-9e17-58f01ed42bfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:27.307414 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ee40ba-1174-475d-9e17-58f01ed42bfd None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:27.307697 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ee40ba-1174-475d-9e17-58f01ed42bfd None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:27.308005 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ee40ba-1174-475d-9e17-58f01ed42bfd None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:27.319115 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ee40ba-1174-475d-9e17-58f01ed42bfd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 11:07:27.327735 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-69ee40ba-1174-475d-9e17-58f01ed42bfd None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 29 11:07:27.327735 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:07:27.327735 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:07:27.327735 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:07:27.327735 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:07:27.327735 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:07:27.327735 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:07:27.327735 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:07:27.327735 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:07:27.327735 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:07:27.327735 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:07:27.327735 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:07:27.327735 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 11:07:27.327735 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 11:07:27.327735 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 11:07:27.327735 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 11:07:27.328730 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 11:07:27.328730 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 11:07:27.328730 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 11:07:27.328730 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 11:07:27.328730 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:07:27.328730 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:07:27.328730 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:07:27.328730 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:07:27.328730 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:07:27.328730 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:07:27.328730 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:07:27.328730 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jun 29 11:07:27.328730 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:07:27.328730 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:07:27.328730 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:07:27.328730 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:07:27.330009 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:07:27.330009 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:07:27.330009 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:07:27.330009 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:07:27.330009 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 11:07:27.330009 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 11:07:27.330009 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 11:07:27.330009 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 11:07:27.330009 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 11:07:27.330009 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 11:07:27.330009 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 29 11:07:27.330009 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:07:27.330009 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 460/921] 10.222.0.44 () {56 vars in 1120 bytes} [Wed Jun 29 11:07:27 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:07:27.337148 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d0d0a3-f2d5-4479-a544-3491e30eaefa None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:27.457488 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d0d0a3-f2d5-4479-a544-3491e30eaefa None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:27.470964 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:27.493128 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13301dc6-e60b-42e8-9feb-254e313e14b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:27.493855 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13301dc6-e60b-42e8-9feb-254e313e14b6 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:27.494194 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13301dc6-e60b-42e8-9feb-254e313e14b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:27.494524 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13301dc6-e60b-42e8-9feb-254e313e14b6 None admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:27.495281 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13301dc6-e60b-42e8-9feb-254e313e14b6 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:07:27.495647 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13301dc6-e60b-42e8-9feb-254e313e14b6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:27.497057 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13301dc6-e60b-42e8-9feb-254e313e14b6 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:27.505743 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 462/922] 10.222.0.44 () {56 vars in 1115 bytes} [Wed Jun 29 11:07:27 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:07:27.511535 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ee40ba-1174-475d-9e17-58f01ed42bfd None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:27.624124 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ee40ba-1174-475d-9e17-58f01ed42bfd None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:27.633718 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b76981-e82f-4b77-9c5a-1fb1a8e1ffab None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:27.634015 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b76981-e82f-4b77-9c5a-1fb1a8e1ffab None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:27.634294 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b76981-e82f-4b77-9c5a-1fb1a8e1ffab None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:27.635807 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 463/923] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:07:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 29 11:07:27.640860 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554489c9-8281-4c95-a0e4-535a290cb85b None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:27.641151 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554489c9-8281-4c95-a0e4-535a290cb85b None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:27.641435 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554489c9-8281-4c95-a0e4-535a290cb85b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:27.641581 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:27.659964 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-554489c9-8281-4c95-a0e4-535a290cb85b None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:07:27.661259 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-554489c9-8281-4c95-a0e4-535a290cb85b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:07:27.662595 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054cdf01-6b3e-45a2-b4b0-be19e9e12d62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:27.663782 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054cdf01-6b3e-45a2-b4b0-be19e9e12d62 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:27.665153 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054cdf01-6b3e-45a2-b4b0-be19e9e12d62 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:27.665634 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054cdf01-6b3e-45a2-b4b0-be19e9e12d62 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:27.679349 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054cdf01-6b3e-45a2-b4b0-be19e9e12d62 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:27.681192 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054cdf01-6b3e-45a2-b4b0-be19e9e12d62 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:27.704189 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-054cdf01-6b3e-45a2-b4b0-be19e9e12d62 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 11:07:27.704189 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:07:27.704189 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:07:27.704189 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:07:27.704189 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:07:27.704189 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:07:27.704189 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:07:27.704189 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:07:27.704189 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:07:27.704189 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:07:27.704189 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:07:27.704189 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:07:27.704189 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 11:07:27.704189 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 11:07:27.704189 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 11:07:27.705112 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 11:07:27.705112 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:07:27.705112 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:07:27.705112 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:07:27.705112 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:07:27.705112 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:07:27.705112 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:07:27.705112 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:07:27.705112 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jun 29 11:07:27.705112 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:07:27.705112 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:07:27.705112 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:07:27.705112 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:07:27.705112 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:07:27.705112 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:07:27.705112 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:07:27.705112 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:07:27.705992 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 11:07:27.705992 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 11:07:27.705992 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 11:07:27.705992 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:07:27.708902 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 461/924] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jun 29 11:07:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:07:27.714201 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054cdf01-6b3e-45a2-b4b0-be19e9e12d62 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:27.740037 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 464/925] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:07:27 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 29 11:07:27.769541 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554489c9-8281-4c95-a0e4-535a290cb85b None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:27.831929 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054cdf01-6b3e-45a2-b4b0-be19e9e12d62 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:27.868286 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b7aecd-583f-45d7-a508-e7014673a4f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:27.869075 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b7aecd-583f-45d7-a508-e7014673a4f3 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:27.869358 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b7aecd-583f-45d7-a508-e7014673a4f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:27.869636 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b7aecd-583f-45d7-a508-e7014673a4f3 None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:27.870470 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b7aecd-583f-45d7-a508-e7014673a4f3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:07:27.870779 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b7aecd-583f-45d7-a508-e7014673a4f3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:27.872174 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b7aecd-583f-45d7-a508-e7014673a4f3 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:27.885776 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 462/926] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Jun 29 11:07:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 11:07:27.891501 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b7aecd-583f-45d7-a508-e7014673a4f3 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:27.898726 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554489c9-8281-4c95-a0e4-535a290cb85b None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:27.901096 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:27.922735 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:27.939708 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e52bdd0-f771-4359-95c8-2c29379e22bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:27.941131 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e52bdd0-f771-4359-95c8-2c29379e22bf None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:27.941476 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e52bdd0-f771-4359-95c8-2c29379e22bf None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:27.941753 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e52bdd0-f771-4359-95c8-2c29379e22bf None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:27.942927 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e52bdd0-f771-4359-95c8-2c29379e22bf None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:27.944824 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e52bdd0-f771-4359-95c8-2c29379e22bf None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:27.962850 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-3e52bdd0-f771-4359-95c8-2c29379e22bf None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 29 11:07:27.962850 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:07:27.962850 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 29 11:07:27.962850 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 29 11:07:27.962850 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 29 11:07:27.962850 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 29 11:07:27.962850 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 29 11:07:27.962850 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 11:07:27.962850 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 29 11:07:27.962850 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jun 29 11:07:27.962850 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 29 11:07:27.962850 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jun 29 11:07:27.962850 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:07:27.962850 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:07:27.962850 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:07:27.962850 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:07:27.963933 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:07:27.963933 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:07:27.963933 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:07:27.963933 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:07:27.963933 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:07:27.963933 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:07:27.963933 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:07:27.963933 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 29 11:07:27.963933 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 29 11:07:27.963933 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 29 11:07:27.963933 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 29 11:07:27.963933 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:07:27.963933 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:07:27.963933 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:07:27.963933 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:07:27.963933 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:07:27.964858 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:07:27.964858 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:07:27.964858 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jun 29 11:07:27.964858 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:07:27.964858 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:07:27.964858 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:07:27.964858 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:07:27.964858 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:07:27.964858 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:07:27.964858 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:07:27.964858 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:07:27.964858 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 29 11:07:27.964858 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 11:07:27.964858 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 29 11:07:27.964858 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jun 29 11:07:27.969145 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 465/927] 10.222.0.44 () {56 vars in 1132 bytes} [Wed Jun 29 11:07:27 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:07:27.970004 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e52bdd0-f771-4359-95c8-2c29379e22bf None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:28.010452 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b7aecd-583f-45d7-a508-e7014673a4f3 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:28.055100 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53aa8f2-3ccc-4b1c-a25e-b359597c92f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:28.055941 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53aa8f2-3ccc-4b1c-a25e-b359597c92f8 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:28.056229 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53aa8f2-3ccc-4b1c-a25e-b359597c92f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:28.056489 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53aa8f2-3ccc-4b1c-a25e-b359597c92f8 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:28.063948 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53aa8f2-3ccc-4b1c-a25e-b359597c92f8 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:28.065363 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53aa8f2-3ccc-4b1c-a25e-b359597c92f8 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:28.081142 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-f53aa8f2-3ccc-4b1c-a25e-b359597c92f8 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 29 11:07:28.081142 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:07:28.081142 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:07:28.081142 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:07:28.081142 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:07:28.081142 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:07:28.081142 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:07:28.081142 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:07:28.081142 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:07:28.081142 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:07:28.081142 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:07:28.081142 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:07:28.081142 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 11:07:28.081142 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 11:07:28.081142 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 11:07:28.081142 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 11:07:28.082123 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:07:28.082123 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:07:28.082123 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 11:07:28.082123 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 11:07:28.082123 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 11:07:28.082123 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 11:07:28.082123 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:07:28.082123 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:07:28.082123 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:07:28.082123 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:07:28.082123 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:07:28.082123 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jun 29 11:07:28.082123 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:07:28.082123 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:07:28.082123 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:07:28.082123 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:07:28.082123 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:07:28.082123 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:07:28.083063 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:07:28.083063 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:07:28.083063 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 11:07:28.083063 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 11:07:28.083063 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 11:07:28.083063 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 11:07:28.083063 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 11:07:28.083063 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 11:07:28.083063 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 29 11:07:28.083063 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:07:28.083063 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 463/928] 10.222.0.44 () {56 vars in 1117 bytes} [Wed Jun 29 11:07:27 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:07:28.088377 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53aa8f2-3ccc-4b1c-a25e-b359597c92f8 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:28.095000 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e52bdd0-f771-4359-95c8-2c29379e22bf None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:28.130527 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e78b60d-7bd8-4601-95a9-1b0cf4a1be2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:28.131487 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e78b60d-7bd8-4601-95a9-1b0cf4a1be2f None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:28.132011 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e78b60d-7bd8-4601-95a9-1b0cf4a1be2f None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:28.132493 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e78b60d-7bd8-4601-95a9-1b0cf4a1be2f None admin] PATH_INFO: `/v3/services` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:28.133375 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e78b60d-7bd8-4601-95a9-1b0cf4a1be2f None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:07:28.133888 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e78b60d-7bd8-4601-95a9-1b0cf4a1be2f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:28.135457 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e78b60d-7bd8-4601-95a9-1b0cf4a1be2f None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:28.140866 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 466/929] 10.222.0.44 () {56 vars in 1126 bytes} [Wed Jun 29 11:07:27 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:07:28.151864 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e78b60d-7bd8-4601-95a9-1b0cf4a1be2f None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:28.185896 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:28.194299 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:28.199192 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53aa8f2-3ccc-4b1c-a25e-b359597c92f8 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:28.249159 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b2a42d-0b75-4031-8be7-b8a2ac75c18e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:28.249913 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b2a42d-0b75-4031-8be7-b8a2ac75c18e None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:28.250359 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b2a42d-0b75-4031-8be7-b8a2ac75c18e None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:28.250639 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b2a42d-0b75-4031-8be7-b8a2ac75c18e None admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:28.252006 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b2a42d-0b75-4031-8be7-b8a2ac75c18e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:07:28.252356 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b2a42d-0b75-4031-8be7-b8a2ac75c18e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:28.254401 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b2a42d-0b75-4031-8be7-b8a2ac75c18e None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:28.275646 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 464/930] 10.222.0.44 () {56 vars in 1149 bytes} [Wed Jun 29 11:07:28 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 11:07:28.278786 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e78b60d-7bd8-4601-95a9-1b0cf4a1be2f None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:28.281300 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b2a42d-0b75-4031-8be7-b8a2ac75c18e None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:28.330347 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f535953-a454-4957-858c-114e22eaa204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:28.331791 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f535953-a454-4957-858c-114e22eaa204 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:28.332608 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f535953-a454-4957-858c-114e22eaa204 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:28.333105 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f535953-a454-4957-858c-114e22eaa204 None admin] PATH_INFO: `/v3/services` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:28.334022 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f535953-a454-4957-858c-114e22eaa204 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:07:28.334664 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f535953-a454-4957-858c-114e22eaa204 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:28.336440 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f535953-a454-4957-858c-114e22eaa204 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:28.343684 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 467/931] 10.222.0.44 () {56 vars in 1126 bytes} [Wed Jun 29 11:07:28 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:07:28.350921 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f535953-a454-4957-858c-114e22eaa204 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:28.403445 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b2a42d-0b75-4031-8be7-b8a2ac75c18e None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:28.445959 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c078734d-40d9-48ef-8a0c-2277acf740a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:28.447079 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c078734d-40d9-48ef-8a0c-2277acf740a0 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:28.447420 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c078734d-40d9-48ef-8a0c-2277acf740a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:28.447803 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c078734d-40d9-48ef-8a0c-2277acf740a0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:28.458914 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c078734d-40d9-48ef-8a0c-2277acf740a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:28.460585 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c078734d-40d9-48ef-8a0c-2277acf740a0 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:28.465421 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:28.469231 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:28.472600 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f535953-a454-4957-858c-114e22eaa204 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:28.483542 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-c078734d-40d9-48ef-8a0c-2277acf740a0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 11:07:28.483542 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:07:28.483542 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:07:28.483542 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:07:28.483542 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:07:28.483542 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:07:28.483542 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:07:28.483542 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:07:28.483542 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:07:28.483542 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:07:28.483542 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:07:28.483542 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:07:28.483542 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 11:07:28.483542 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 11:07:28.483542 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 11:07:28.484481 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 11:07:28.484481 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:07:28.484481 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:07:28.484481 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:07:28.484481 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:07:28.484481 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:07:28.484481 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:07:28.484481 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:07:28.484481 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jun 29 11:07:28.484481 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:07:28.484481 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:07:28.484481 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:07:28.484481 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:07:28.484481 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:07:28.484481 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:07:28.484481 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:07:28.484481 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:07:28.485394 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 11:07:28.485394 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 11:07:28.485394 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 11:07:28.485394 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:07:28.486283 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 465/932] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jun 29 11:07:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:07:28.491915 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c078734d-40d9-48ef-8a0c-2277acf740a0 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:28.514396 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068ee0fc-2a85-4b1b-bf7e-7a52e26b0214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:28.514993 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068ee0fc-2a85-4b1b-bf7e-7a52e26b0214 None admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:28.515277 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068ee0fc-2a85-4b1b-bf7e-7a52e26b0214 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:28.515666 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068ee0fc-2a85-4b1b-bf7e-7a52e26b0214 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:28.516470 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068ee0fc-2a85-4b1b-bf7e-7a52e26b0214 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:28.517947 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068ee0fc-2a85-4b1b-bf7e-7a52e26b0214 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:28.603398 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c078734d-40d9-48ef-8a0c-2277acf740a0 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:28.615887 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 468/933] 10.222.0.44 () {60 vars in 1144 bytes} [Wed Jun 29 11:07:28 2022] POST /identity/v3/endpoints => generated 347 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:07:28.655356 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd5702c-c2d4-4fb3-af2f-c352b6ef2c7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:28.656064 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd5702c-c2d4-4fb3-af2f-c352b6ef2c7e None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:28.656401 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd5702c-c2d4-4fb3-af2f-c352b6ef2c7e None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:28.656681 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd5702c-c2d4-4fb3-af2f-c352b6ef2c7e None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:28.657411 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd5702c-c2d4-4fb3-af2f-c352b6ef2c7e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:07:28.657721 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd5702c-c2d4-4fb3-af2f-c352b6ef2c7e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:28.659225 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd5702c-c2d4-4fb3-af2f-c352b6ef2c7e None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:28.674345 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 466/934] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Jun 29 11:07:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 11:07:28.680682 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068ee0fc-2a85-4b1b-bf7e-7a52e26b0214 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:28.793491 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068ee0fc-2a85-4b1b-bf7e-7a52e26b0214 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:28.833325 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cf2847-595d-488b-9db9-c2b4a49468fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:28.834467 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cf2847-595d-488b-9db9-c2b4a49468fa None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:28.834973 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cf2847-595d-488b-9db9-c2b4a49468fa None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:28.835447 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cf2847-595d-488b-9db9-c2b4a49468fa None admin] PATH_INFO: `/v3/projects/service` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:28.842780 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cf2847-595d-488b-9db9-c2b4a49468fa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:28.844490 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cf2847-595d-488b-9db9-c2b4a49468fa None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:28.859239 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-34cf2847-595d-488b-9db9-c2b4a49468fa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 11:07:28.859239 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:07:28.859239 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:07:28.859239 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:07:28.859239 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:07:28.859239 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:07:28.859239 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:07:28.859239 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:07:28.859239 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:07:28.859239 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:07:28.859239 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:07:28.859239 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:07:28.859239 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 11:07:28.859239 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 11:07:28.859239 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 11:07:28.860216 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 11:07:28.860216 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:07:28.860216 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:07:28.860216 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:07:28.860216 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:07:28.860216 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:07:28.860216 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:07:28.860216 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:07:28.860216 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jun 29 11:07:28.860216 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:07:28.860216 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:07:28.860216 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:07:28.860216 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:07:28.860216 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:07:28.860216 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:07:28.860216 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:07:28.860216 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:07:28.861327 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 11:07:28.861327 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 11:07:28.861327 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 11:07:28.861327 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 11:07:28.861327 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 11:07:28.861327 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 11:07:28.861327 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 11:07:28.861327 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jun 29 11:07:28.863996 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 469/935] 10.222.0.44 () {56 vars in 1111 bytes} [Wed Jun 29 11:07:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:07:28.875853 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd5702c-c2d4-4fb3-af2f-c352b6ef2c7e None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:28.931451 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:28.978329 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd5702c-c2d4-4fb3-af2f-c352b6ef2c7e None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:29.015582 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b28bba-3e30-49d1-8864-5ce275da59a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:29.016282 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b28bba-3e30-49d1-8864-5ce275da59a1 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:29.016569 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b28bba-3e30-49d1-8864-5ce275da59a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:29.016845 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b28bba-3e30-49d1-8864-5ce275da59a1 None admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:29.017698 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b28bba-3e30-49d1-8864-5ce275da59a1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:07:29.018010 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b28bba-3e30-49d1-8864-5ce275da59a1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:29.019467 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b28bba-3e30-49d1-8864-5ce275da59a1 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:29.033809 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 467/936] 10.222.0.44 () {56 vars in 1148 bytes} [Wed Jun 29 11:07:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:07:29.039613 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cf2847-595d-488b-9db9-c2b4a49468fa None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:29.055642 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:29.146928 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cf2847-595d-488b-9db9-c2b4a49468fa None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:29.182181 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11582ec2-0c33-4682-b5d2-39bea0d5836b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:29.183079 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11582ec2-0c33-4682-b5d2-39bea0d5836b None admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:29.183364 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11582ec2-0c33-4682-b5d2-39bea0d5836b None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:29.183639 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11582ec2-0c33-4682-b5d2-39bea0d5836b None admin] PATH_INFO: `/v3/projects/ebe5dffa4bb8440286687daefb6c98f4/users/55487c2107a44c15b656b4865ee0d4fa/roles/3e8ef91b147f4b6b803ef07f3301be3f` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:29.185892 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11582ec2-0c33-4682-b5d2-39bea0d5836b None admin] RBAC: Authorizing `identity:create_grant(project_id=ebe5dffa4bb8440286687daefb6c98f4, user_id=55487c2107a44c15b656b4865ee0d4fa, role_id=3e8ef91b147f4b6b803ef07f3301be3f)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:29.187528 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11582ec2-0c33-4682-b5d2-39bea0d5836b None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:29.214318 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2358c5-946b-42cc-8c3b-e84818a1238e None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:29.214603 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2358c5-946b-42cc-8c3b-e84818a1238e None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:29.214878 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2358c5-946b-42cc-8c3b-e84818a1238e None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:29.216419 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 468/937] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:07:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:07:29.221066 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612e3b16-cdbe-4968-be25-aa4d0c8994b1 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:29.221404 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612e3b16-cdbe-4968-be25-aa4d0c8994b1 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:29.222265 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612e3b16-cdbe-4968-be25-aa4d0c8994b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:29.239865 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-612e3b16-cdbe-4968-be25-aa4d0c8994b1 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:07:29.244634 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-612e3b16-cdbe-4968-be25-aa4d0c8994b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:07:29.279873 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 470/938] 10.222.0.44 () {58 vars in 1439 bytes} [Wed Jun 29 11:07:29 2022] PUT /identity/v3/projects/ebe5dffa4bb8440286687daefb6c98f4/users/55487c2107a44c15b656b4865ee0d4fa/roles/3e8ef91b147f4b6b803ef07f3301be3f => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:07:29.319396 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 469/939] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:07:29 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 29 11:07:29.343592 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11582ec2-0c33-4682-b5d2-39bea0d5836b None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:29.425664 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:29.464852 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11582ec2-0c33-4682-b5d2-39bea0d5836b None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:29.475252 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c733e0-7ad0-44db-8737-5695bbc049ed None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:29.480899 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c733e0-7ad0-44db-8737-5695bbc049ed None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:29.480899 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c733e0-7ad0-44db-8737-5695bbc049ed None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:29.480899 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 470/940] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:07:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:07:29.482692 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f218ba76-0717-4d09-812b-fa38c91d27a1 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:29.483153 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f218ba76-0717-4d09-812b-fa38c91d27a1 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:29.483437 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f218ba76-0717-4d09-812b-fa38c91d27a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:29.508089 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-f218ba76-0717-4d09-812b-fa38c91d27a1 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:07:29.508297 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f3c1fc-a9ce-4c4d-882f-e8dddd0617ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:29.509079 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f3c1fc-a9ce-4c4d-882f-e8dddd0617ed None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:29.509342 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-f218ba76-0717-4d09-812b-fa38c91d27a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:07:29.509450 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f3c1fc-a9ce-4c4d-882f-e8dddd0617ed None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:29.509674 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f3c1fc-a9ce-4c4d-882f-e8dddd0617ed None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:29.510329 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f3c1fc-a9ce-4c4d-882f-e8dddd0617ed None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:29.511693 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f3c1fc-a9ce-4c4d-882f-e8dddd0617ed None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:29.524545 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-a4f3c1fc-a9ce-4c4d-882f-e8dddd0617ed None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 29 11:07:29.524545 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:07:29.524545 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 29 11:07:29.524545 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 29 11:07:29.524545 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 29 11:07:29.524545 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 29 11:07:29.524545 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 29 11:07:29.524545 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 11:07:29.524545 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 29 11:07:29.524545 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jun 29 11:07:29.524545 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 29 11:07:29.524545 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jun 29 11:07:29.524545 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:07:29.524545 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:07:29.524545 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:07:29.524545 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:07:29.524545 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:07:29.525563 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:07:29.525563 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:07:29.525563 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:07:29.525563 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:07:29.525563 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:07:29.525563 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:07:29.525563 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 29 11:07:29.525563 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 29 11:07:29.525563 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 29 11:07:29.525563 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 29 11:07:29.525563 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:07:29.525563 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:07:29.525563 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:07:29.525563 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:07:29.525563 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:07:29.525563 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:07:29.526805 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:07:29.526805 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jun 29 11:07:29.526805 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:07:29.526805 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:07:29.526805 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:07:29.526805 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:07:29.526805 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:07:29.526805 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:07:29.526805 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:07:29.526805 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:07:29.526805 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 29 11:07:29.526805 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 11:07:29.526805 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 29 11:07:29.526805 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jun 29 11:07:29.526805 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 471/941] 10.222.0.44 () {56 vars in 1126 bytes} [Wed Jun 29 11:07:29 2022] GET /identity/v3/services/object-store => generated 93 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:07:29.531720 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f3c1fc-a9ce-4c4d-882f-e8dddd0617ed None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:29.553977 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:29.588172 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 471/942] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:07:29 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 29 11:07:29.614414 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f218ba76-0717-4d09-812b-fa38c91d27a1 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:29.645072 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f3c1fc-a9ce-4c4d-882f-e8dddd0617ed None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:29.682367 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3d5fd1-7f54-4cdb-8de0-f49b6a00bc69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:29.683136 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3d5fd1-7f54-4cdb-8de0-f49b6a00bc69 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:29.683419 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3d5fd1-7f54-4cdb-8de0-f49b6a00bc69 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:29.683695 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3d5fd1-7f54-4cdb-8de0-f49b6a00bc69 None admin] PATH_INFO: `/v3/services` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:29.684430 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3d5fd1-7f54-4cdb-8de0-f49b6a00bc69 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:07:29.684799 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3d5fd1-7f54-4cdb-8de0-f49b6a00bc69 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:29.686263 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3d5fd1-7f54-4cdb-8de0-f49b6a00bc69 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:29.689580 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:29.694527 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 472/943] 10.222.0.44 () {56 vars in 1122 bytes} [Wed Jun 29 11:07:29 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:07:29.700799 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3d5fd1-7f54-4cdb-8de0-f49b6a00bc69 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:29.719974 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f218ba76-0717-4d09-812b-fa38c91d27a1 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:29.757345 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d322f77-6066-4afb-ad00-e0371f96ec50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:29.758015 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d322f77-6066-4afb-ad00-e0371f96ec50 None admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:29.758297 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d322f77-6066-4afb-ad00-e0371f96ec50 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:29.758573 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d322f77-6066-4afb-ad00-e0371f96ec50 None admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:29.759288 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d322f77-6066-4afb-ad00-e0371f96ec50 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:29.760747 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d322f77-6066-4afb-ad00-e0371f96ec50 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:29.762121 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: WARNING keystone.server.flask.common [None req-7d322f77-6066-4afb-ad00-e0371f96ec50 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jun 29 11:07:29.809731 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3d5fd1-7f54-4cdb-8de0-f49b6a00bc69 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:29.825361 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.sql.core [None req-7d322f77-6066-4afb-ad00-e0371f96ec50 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 29 11:07:29.825361 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 29 11:07:29.825361 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [parameters: {'id': 'a35affab550e47238e494845f373bfea', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 29 11:07:29.825361 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80166) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 29 11:07:29.845659 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde46cd9-14c5-4115-b5d1-5c94daa7f5d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:29.846626 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde46cd9-14c5-4115-b5d1-5c94daa7f5d8 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:29.847612 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde46cd9-14c5-4115-b5d1-5c94daa7f5d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:29.847717 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-7d322f77-6066-4afb-ad00-e0371f96ec50 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id a35affab550e47238e494845f373bfea.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id a35affab550e47238e494845f373bfea. Jun 29 11:07:29.847717 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:07:29.847717 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 29 11:07:29.847717 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 29 11:07:29.847717 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 29 11:07:29.847717 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 29 11:07:29.847717 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 29 11:07:29.847717 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application result = self._query(query) Jun 29 11:07:29.847717 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 29 11:07:29.847717 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application conn.query(q) Jun 29 11:07:29.847717 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 29 11:07:29.847717 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 29 11:07:29.847717 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 29 11:07:29.848506 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application result.read() Jun 29 11:07:29.848506 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 29 11:07:29.848506 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 29 11:07:29.848506 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 29 11:07:29.848506 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 29 11:07:29.848506 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 29 11:07:29.848506 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 29 11:07:29.848506 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 29 11:07:29.848506 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 29 11:07:29.848506 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 29 11:07:29.848506 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:07:29.848506 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 29 11:07:29.848506 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:07:29.848506 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:07:29.848506 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 29 11:07:29.848506 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 29 11:07:29.848506 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jun 29 11:07:29.848506 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return project_ref.to_dict() Jun 29 11:07:29.849714 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 29 11:07:29.849714 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application next(self.gen) Jun 29 11:07:29.849714 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 29 11:07:29.849714 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application yield resource Jun 29 11:07:29.849714 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 29 11:07:29.849714 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application next(self.gen) Jun 29 11:07:29.849714 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 29 11:07:29.849714 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self.session.rollback() Jun 29 11:07:29.849714 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 29 11:07:29.849714 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self.force_reraise() Jun 29 11:07:29.849714 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 29 11:07:29.849714 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise self.value Jun 29 11:07:29.849714 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 29 11:07:29.849714 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 29 11:07:29.849714 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 29 11:07:29.849714 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application session.commit() Jun 29 11:07:29.849714 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 29 11:07:29.850631 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 29 11:07:29.850631 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 29 11:07:29.850631 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self._prepare_impl() Jun 29 11:07:29.850631 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 29 11:07:29.850631 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self.session.flush() Jun 29 11:07:29.850631 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 29 11:07:29.850631 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self._flush(objects) Jun 29 11:07:29.850631 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 29 11:07:29.850631 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 29 11:07:29.850631 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 29 11:07:29.850631 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application compat.raise_( Jun 29 11:07:29.850631 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 29 11:07:29.850631 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception Jun 29 11:07:29.850631 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 29 11:07:29.850631 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application flush_context.execute() Jun 29 11:07:29.850631 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 29 11:07:29.850631 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rec.execute(self) Jun 29 11:07:29.851557 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 29 11:07:29.851557 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 29 11:07:29.851557 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 29 11:07:29.851557 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 29 11:07:29.851557 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 29 11:07:29.851557 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 29 11:07:29.851557 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 29 11:07:29.851557 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 29 11:07:29.851557 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jun 29 11:07:29.851557 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 29 11:07:29.851557 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 29 11:07:29.851557 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 29 11:07:29.851557 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 29 11:07:29.851557 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 29 11:07:29.851557 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 29 11:07:29.852414 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 29 11:07:29.852414 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 29 11:07:29.852414 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception Jun 29 11:07:29.852414 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 29 11:07:29.852414 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 29 11:07:29.852414 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 29 11:07:29.852414 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 29 11:07:29.852414 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 29 11:07:29.852414 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application result = self._query(query) Jun 29 11:07:29.852414 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 29 11:07:29.852414 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application conn.query(q) Jun 29 11:07:29.852414 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 29 11:07:29.852414 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 29 11:07:29.852414 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 29 11:07:29.852414 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application result.read() Jun 29 11:07:29.852414 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 29 11:07:29.852414 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 29 11:07:29.853287 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 29 11:07:29.853287 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 29 11:07:29.853287 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 29 11:07:29.853287 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 29 11:07:29.853287 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 29 11:07:29.853287 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 29 11:07:29.853287 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 29 11:07:29.853287 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 29 11:07:29.853287 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [parameters: {'id': 'a35affab550e47238e494845f373bfea', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 29 11:07:29.853287 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 29 11:07:29.853287 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:07:29.853287 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 29 11:07:29.853287 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:07:29.853287 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:07:29.853287 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jun 29 11:07:29.854196 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jun 29 11:07:29.854196 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 29 11:07:29.854196 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 29 11:07:29.854196 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jun 29 11:07:29.854196 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:07:29.854196 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 29 11:07:29.854196 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:07:29.854196 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:07:29.854196 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:07:29.854196 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:07:29.854196 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:07:29.854196 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:07:29.854196 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:07:29.854196 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:07:29.854196 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:07:29.854196 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:07:29.854196 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:07:29.855165 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:07:29.855165 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jun 29 11:07:29.855165 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jun 29 11:07:29.855165 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:07:29.855165 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:07:29.855165 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jun 29 11:07:29.855165 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.Conflict( Jun 29 11:07:29.855165 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id a35affab550e47238e494845f373bfea. Jun 29 11:07:29.855165 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:07:29.855165 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 472/944] 10.222.0.44 () {60 vars in 1140 bytes} [Wed Jun 29 11:07:29 2022] POST /identity/v3/projects => generated 262 bytes in 240 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:07:29.855737 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d322f77-6066-4afb-ad00-e0371f96ec50 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:29.858706 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde46cd9-14c5-4115-b5d1-5c94daa7f5d8 None admin] PATH_INFO: `/v3/services` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:29.858706 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde46cd9-14c5-4115-b5d1-5c94daa7f5d8 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:07:29.858706 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde46cd9-14c5-4115-b5d1-5c94daa7f5d8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:29.861396 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde46cd9-14c5-4115-b5d1-5c94daa7f5d8 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:29.868469 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 473/945] 10.222.0.44 () {56 vars in 1122 bytes} [Wed Jun 29 11:07:29 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:07:29.875139 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde46cd9-14c5-4115-b5d1-5c94daa7f5d8 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:29.896688 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:29.946204 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:29.973156 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d322f77-6066-4afb-ad00-e0371f96ec50 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:29.988804 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde46cd9-14c5-4115-b5d1-5c94daa7f5d8 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:30.010279 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2423d5-0674-4d74-ab46-19a3413e75a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:30.011029 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2423d5-0674-4d74-ab46-19a3413e75a8 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:30.011737 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2423d5-0674-4d74-ab46-19a3413e75a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:30.012399 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2423d5-0674-4d74-ab46-19a3413e75a8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:30.020527 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2423d5-0674-4d74-ab46-19a3413e75a8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:30.022439 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2423d5-0674-4d74-ab46-19a3413e75a8 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:30.027553 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bcb7e8-668e-4ee0-86ed-93b7bb8f0060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:30.028429 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bcb7e8-668e-4ee0-86ed-93b7bb8f0060 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:30.028962 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bcb7e8-668e-4ee0-86ed-93b7bb8f0060 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:30.029382 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bcb7e8-668e-4ee0-86ed-93b7bb8f0060 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:30.030363 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bcb7e8-668e-4ee0-86ed-93b7bb8f0060 None admin] RBAC: Adding query filter params (interface=public, service_id=99486dc5d2414777a654e34cc11881c2, region_id=RegionOne) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:07:30.030831 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bcb7e8-668e-4ee0-86ed-93b7bb8f0060 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:30.032488 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bcb7e8-668e-4ee0-86ed-93b7bb8f0060 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:30.037880 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-db2423d5-0674-4d74-ab46-19a3413e75a8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 11:07:30.037880 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:07:30.037880 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:07:30.037880 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:07:30.037880 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:07:30.037880 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:07:30.037880 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:07:30.037880 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:07:30.037880 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:07:30.037880 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:07:30.037880 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:07:30.037880 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:07:30.037880 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 11:07:30.037880 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 11:07:30.037880 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 11:07:30.038979 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 11:07:30.038979 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:07:30.038979 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:07:30.038979 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:07:30.038979 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:07:30.038979 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:07:30.038979 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:07:30.038979 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:07:30.038979 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jun 29 11:07:30.038979 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:07:30.038979 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:07:30.038979 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:07:30.038979 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:07:30.038979 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:07:30.038979 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:07:30.038979 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:07:30.038979 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:07:30.040163 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 11:07:30.040163 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 11:07:30.040163 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 11:07:30.040163 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 11:07:30.040163 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 11:07:30.040163 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 11:07:30.040163 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 11:07:30.040163 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:07:30.048133 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 473/946] 10.222.0.44 () {56 vars in 1111 bytes} [Wed Jun 29 11:07:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:07:30.048279 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 474/947] 10.222.0.44 () {56 vars in 1251 bytes} [Wed Jun 29 11:07:29 2022] GET /identity/v3/endpoints?service_id=99486dc5d2414777a654e34cc11881c2&interface=public®ion_id=RegionOne => generated 194 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:07:30.050706 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2423d5-0674-4d74-ab46-19a3413e75a8 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:30.052647 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bcb7e8-668e-4ee0-86ed-93b7bb8f0060 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:30.148840 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:30.158713 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2423d5-0674-4d74-ab46-19a3413e75a8 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:30.161492 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bcb7e8-668e-4ee0-86ed-93b7bb8f0060 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:30.191092 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:30.199445 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c4623c-49a4-47dd-9c71-a5daa4854fdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:30.200050 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f55ddb-211d-4b25-b4d5-2b598f4a1882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:30.200535 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c4623c-49a4-47dd-9c71-a5daa4854fdd None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:30.201332 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f55ddb-211d-4b25-b4d5-2b598f4a1882 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:30.201450 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c4623c-49a4-47dd-9c71-a5daa4854fdd None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:30.201552 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f55ddb-211d-4b25-b4d5-2b598f4a1882 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:30.201651 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c4623c-49a4-47dd-9c71-a5daa4854fdd None admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:30.201900 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f55ddb-211d-4b25-b4d5-2b598f4a1882 None admin] PATH_INFO: `/v3/services` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:30.202619 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c4623c-49a4-47dd-9c71-a5daa4854fdd None admin] RBAC: Adding query filter params (name=service) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:07:30.202740 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f55ddb-211d-4b25-b4d5-2b598f4a1882 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:30.203181 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c4623c-49a4-47dd-9c71-a5daa4854fdd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:30.204332 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f55ddb-211d-4b25-b4d5-2b598f4a1882 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:30.204941 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c4623c-49a4-47dd-9c71-a5daa4854fdd None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:30.218391 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 475/948] 10.222.0.44 () {56 vars in 1087 bytes} [Wed Jun 29 11:07:30 2022] GET /identity/v3/services => generated 1497 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 11:07:30.229088 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 474/949] 10.222.0.44 () {56 vars in 1112 bytes} [Wed Jun 29 11:07:30 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:07:30.729355 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f120e501-b72b-4299-905a-03072339a3b0 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:30.729870 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f120e501-b72b-4299-905a-03072339a3b0 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:30.729948 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f120e501-b72b-4299-905a-03072339a3b0 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:30.731385 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 476/950] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:07:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:07:30.736823 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f293679-11b3-4f72-94f2-d29cd29ce1d4 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:30.737638 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f293679-11b3-4f72-94f2-d29cd29ce1d4 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:30.738154 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f293679-11b3-4f72-94f2-d29cd29ce1d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:30.756113 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-2f293679-11b3-4f72-94f2-d29cd29ce1d4 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:07:30.757930 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-2f293679-11b3-4f72-94f2-d29cd29ce1d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:07:30.826210 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 475/951] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:07:30 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 29 11:07:30.849091 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f120e501-b72b-4299-905a-03072339a3b0 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:30.954523 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f120e501-b72b-4299-905a-03072339a3b0 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:30.990218 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f450da3e-43d0-40ab-8ee9-cacba155fa1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:30.990933 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f450da3e-43d0-40ab-8ee9-cacba155fa1c None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:30.991214 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f450da3e-43d0-40ab-8ee9-cacba155fa1c None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:30.991489 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f450da3e-43d0-40ab-8ee9-cacba155fa1c None admin] PATH_INFO: `/v3/services/compute` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:30.992164 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f450da3e-43d0-40ab-8ee9-cacba155fa1c None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:30.993534 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f450da3e-43d0-40ab-8ee9-cacba155fa1c None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:31.006211 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-f450da3e-43d0-40ab-8ee9-cacba155fa1c None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 29 11:07:31.006211 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:07:31.006211 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 29 11:07:31.006211 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 29 11:07:31.006211 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 29 11:07:31.006211 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 29 11:07:31.006211 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 29 11:07:31.006211 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 11:07:31.006211 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 29 11:07:31.006211 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jun 29 11:07:31.006211 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 29 11:07:31.006211 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jun 29 11:07:31.006211 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:07:31.006211 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:07:31.006211 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:07:31.006211 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:07:31.006211 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:07:31.007279 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:07:31.007279 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:07:31.007279 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:07:31.007279 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:07:31.007279 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:07:31.007279 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:07:31.007279 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 29 11:07:31.007279 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 29 11:07:31.007279 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 29 11:07:31.007279 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 29 11:07:31.007279 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:07:31.007279 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:07:31.007279 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:07:31.007279 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:07:31.007279 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:07:31.007279 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:07:31.008789 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:07:31.008789 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jun 29 11:07:31.008789 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:07:31.008789 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:07:31.008789 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:07:31.008789 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:07:31.008789 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:07:31.008789 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:07:31.008789 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:07:31.008789 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:07:31.008789 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 29 11:07:31.008789 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 11:07:31.008789 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 29 11:07:31.008789 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jun 29 11:07:31.008789 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 477/952] 10.222.0.44 () {56 vars in 1111 bytes} [Wed Jun 29 11:07:30 2022] GET /identity/v3/services/compute => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:07:31.013239 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f293679-11b3-4f72-94f2-d29cd29ce1d4 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:31.094769 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:31.114845 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f293679-11b3-4f72-94f2-d29cd29ce1d4 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:31.152053 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57697c1e-aa6e-454b-bda9-7f21b1b001db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:31.152722 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57697c1e-aa6e-454b-bda9-7f21b1b001db None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:31.153007 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57697c1e-aa6e-454b-bda9-7f21b1b001db None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:31.153287 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57697c1e-aa6e-454b-bda9-7f21b1b001db None admin] PATH_INFO: `/v3/services` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:31.153989 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57697c1e-aa6e-454b-bda9-7f21b1b001db None admin] RBAC: Adding query filter params (name=compute) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:07:31.154298 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57697c1e-aa6e-454b-bda9-7f21b1b001db None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:31.155653 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57697c1e-aa6e-454b-bda9-7f21b1b001db None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:31.160877 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 476/953] 10.222.0.44 () {56 vars in 1112 bytes} [Wed Jun 29 11:07:31 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:07:31.166140 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f450da3e-43d0-40ab-8ee9-cacba155fa1c None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:31.222879 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:31.267570 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f450da3e-43d0-40ab-8ee9-cacba155fa1c None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:31.301802 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31835ebe-7260-4888-8086-c2b5402d7a56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:31.302464 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31835ebe-7260-4888-8086-c2b5402d7a56 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:31.302747 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31835ebe-7260-4888-8086-c2b5402d7a56 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:31.303020 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31835ebe-7260-4888-8086-c2b5402d7a56 None admin] PATH_INFO: `/v3/services` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:31.303719 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31835ebe-7260-4888-8086-c2b5402d7a56 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:07:31.304062 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31835ebe-7260-4888-8086-c2b5402d7a56 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:31.305422 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31835ebe-7260-4888-8086-c2b5402d7a56 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:31.310319 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 478/954] 10.222.0.44 () {56 vars in 1112 bytes} [Wed Jun 29 11:07:31 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:07:31.442068 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fd60d4-a4e4-4747-afad-f0389ddf8c65 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:31.442879 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fd60d4-a4e4-4747-afad-f0389ddf8c65 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:31.443396 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fd60d4-a4e4-4747-afad-f0389ddf8c65 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:31.445486 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 477/955] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:07:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:07:31.452370 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ba61a0-d8de-44b9-9cca-22b9cdd18c80 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:31.452370 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ba61a0-d8de-44b9-9cca-22b9cdd18c80 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:31.452839 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ba61a0-d8de-44b9-9cca-22b9cdd18c80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:31.472399 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-f1ba61a0-d8de-44b9-9cca-22b9cdd18c80 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:07:31.473617 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-f1ba61a0-d8de-44b9-9cca-22b9cdd18c80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:07:31.548388 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 479/956] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:07:31 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 11:07:31.572532 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fd60d4-a4e4-4747-afad-f0389ddf8c65 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:31.681405 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fd60d4-a4e4-4747-afad-f0389ddf8c65 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:31.717731 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11ef04b-ee12-4cf7-a04e-a2dea13600f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:31.718543 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11ef04b-ee12-4cf7-a04e-a2dea13600f6 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:31.718831 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11ef04b-ee12-4cf7-a04e-a2dea13600f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:31.719106 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11ef04b-ee12-4cf7-a04e-a2dea13600f6 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:31.729481 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11ef04b-ee12-4cf7-a04e-a2dea13600f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 11:07:31.738715 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-d11ef04b-ee12-4cf7-a04e-a2dea13600f6 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 29 11:07:31.738715 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:07:31.738715 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:07:31.738715 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:07:31.738715 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:07:31.738715 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:07:31.738715 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:07:31.738715 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:07:31.738715 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:07:31.738715 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:07:31.738715 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:07:31.738715 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:07:31.738715 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 11:07:31.738715 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 11:07:31.738715 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 11:07:31.738715 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 11:07:31.740341 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 11:07:31.740341 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 11:07:31.740341 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 11:07:31.740341 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 11:07:31.740341 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:07:31.740341 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:07:31.740341 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:07:31.740341 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:07:31.740341 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:07:31.740341 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:07:31.740341 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:07:31.740341 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jun 29 11:07:31.740341 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:07:31.740341 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:07:31.740341 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:07:31.740341 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:07:31.741290 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:07:31.741290 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:07:31.741290 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:07:31.741290 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:07:31.741290 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 11:07:31.741290 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 11:07:31.741290 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 11:07:31.741290 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 11:07:31.741290 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 11:07:31.741290 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 11:07:31.741290 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 29 11:07:31.741290 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:07:31.741290 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 478/957] 10.222.0.44 () {56 vars in 1120 bytes} [Wed Jun 29 11:07:31 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:07:31.748556 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ba61a0-d8de-44b9-9cca-22b9cdd18c80 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:31.799427 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:31.850482 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ba61a0-d8de-44b9-9cca-22b9cdd18c80 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:31.887112 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3712c5e-e81a-45c2-a712-2791ba03a1e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:31.888188 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3712c5e-e81a-45c2-a712-2791ba03a1e5 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:31.888695 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3712c5e-e81a-45c2-a712-2791ba03a1e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:31.889269 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3712c5e-e81a-45c2-a712-2791ba03a1e5 None admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:31.890220 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3712c5e-e81a-45c2-a712-2791ba03a1e5 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:07:31.890739 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3712c5e-e81a-45c2-a712-2791ba03a1e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:31.892381 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3712c5e-e81a-45c2-a712-2791ba03a1e5 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:31.905629 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 480/958] 10.222.0.44 () {56 vars in 1115 bytes} [Wed Jun 29 11:07:31 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:07:31.907266 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11ef04b-ee12-4cf7-a04e-a2dea13600f6 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:31.915890 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:32.019589 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11ef04b-ee12-4cf7-a04e-a2dea13600f6 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:32.054923 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa151c1-a74d-4769-abb6-47aaf9583005 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:32.055635 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa151c1-a74d-4769-abb6-47aaf9583005 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:32.055947 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa151c1-a74d-4769-abb6-47aaf9583005 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:32.056223 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa151c1-a74d-4769-abb6-47aaf9583005 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:32.067359 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa151c1-a74d-4769-abb6-47aaf9583005 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:32.068831 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa151c1-a74d-4769-abb6-47aaf9583005 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:32.086896 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-7fa151c1-a74d-4769-abb6-47aaf9583005 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 11:07:32.086896 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:07:32.086896 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:07:32.086896 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:07:32.086896 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:07:32.086896 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:07:32.086896 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:07:32.086896 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:07:32.086896 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:07:32.086896 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:07:32.086896 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:07:32.086896 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:07:32.086896 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 11:07:32.086896 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 11:07:32.086896 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 11:07:32.087911 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 11:07:32.087911 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:07:32.087911 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:07:32.087911 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:07:32.087911 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:07:32.087911 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:07:32.087911 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:07:32.087911 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:07:32.087911 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jun 29 11:07:32.087911 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:07:32.087911 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:07:32.087911 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:07:32.087911 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:07:32.087911 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:07:32.087911 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:07:32.087911 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:07:32.087911 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:07:32.088837 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 11:07:32.088837 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 11:07:32.088837 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 11:07:32.088837 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:07:32.088837 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 479/959] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jun 29 11:07:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:07:32.093987 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3712c5e-e81a-45c2-a712-2791ba03a1e5 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:32.198911 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3712c5e-e81a-45c2-a712-2791ba03a1e5 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:32.212422 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:32.233818 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a434579-100f-49c4-a394-23c248ecd1b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:32.234482 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a434579-100f-49c4-a394-23c248ecd1b0 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:32.237063 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a434579-100f-49c4-a394-23c248ecd1b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:32.237063 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a434579-100f-49c4-a394-23c248ecd1b0 None admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:32.237063 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a434579-100f-49c4-a394-23c248ecd1b0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:07:32.237063 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a434579-100f-49c4-a394-23c248ecd1b0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:32.237563 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a434579-100f-49c4-a394-23c248ecd1b0 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:32.249748 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 481/960] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Jun 29 11:07:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:07:32.255969 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa151c1-a74d-4769-abb6-47aaf9583005 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:32.320340 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:32.373369 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa151c1-a74d-4769-abb6-47aaf9583005 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:32.417603 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baef360d-599b-4625-bdca-e5f2c13a7f19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:32.418504 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baef360d-599b-4625-bdca-e5f2c13a7f19 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:32.418787 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baef360d-599b-4625-bdca-e5f2c13a7f19 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:32.419061 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baef360d-599b-4625-bdca-e5f2c13a7f19 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:32.427145 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baef360d-599b-4625-bdca-e5f2c13a7f19 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:32.428615 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baef360d-599b-4625-bdca-e5f2c13a7f19 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:32.437442 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de75b01b-92cc-4f7d-9b08-9794320fa186 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:32.437735 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de75b01b-92cc-4f7d-9b08-9794320fa186 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:32.438015 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de75b01b-92cc-4f7d-9b08-9794320fa186 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:32.439540 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 482/961] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:07:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:07:32.444276 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-baef360d-599b-4625-bdca-e5f2c13a7f19 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 29 11:07:32.444276 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:07:32.444276 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:07:32.444276 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:07:32.444276 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:07:32.444276 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:07:32.444276 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:07:32.444276 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:07:32.444276 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:07:32.444276 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:07:32.444276 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:07:32.444276 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:07:32.444276 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 11:07:32.444276 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 11:07:32.444276 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 11:07:32.444276 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 11:07:32.445261 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:07:32.445261 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:07:32.445261 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 11:07:32.445261 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 11:07:32.445261 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 11:07:32.445261 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 11:07:32.445261 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:07:32.445261 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:07:32.445261 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:07:32.445261 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:07:32.445261 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:07:32.445261 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jun 29 11:07:32.445261 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:07:32.445261 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:07:32.445261 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:07:32.445261 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:07:32.445261 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:07:32.445261 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:07:32.446633 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:07:32.446633 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:07:32.446633 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 11:07:32.446633 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 11:07:32.446633 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 11:07:32.446633 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 11:07:32.446633 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 11:07:32.446633 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 11:07:32.446633 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 29 11:07:32.446633 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:07:32.447227 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e3769c-990e-48d6-b2a0-b93c039d032f None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:32.447227 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e3769c-990e-48d6-b2a0-b93c039d032f None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:32.447373 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 480/962] 10.222.0.44 () {56 vars in 1117 bytes} [Wed Jun 29 11:07:32 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:07:32.447450 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e3769c-990e-48d6-b2a0-b93c039d032f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:32.454230 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baef360d-599b-4625-bdca-e5f2c13a7f19 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:32.465596 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-53e3769c-990e-48d6-b2a0-b93c039d032f None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:07:32.467246 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-53e3769c-990e-48d6-b2a0-b93c039d032f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:07:32.537698 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 483/963] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:07:32 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (3 switches on core 0) Jun 29 11:07:32.541197 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bc71e8-b176-44dd-a2ff-dea09229e268 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:32.541490 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bc71e8-b176-44dd-a2ff-dea09229e268 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:32.541768 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bc71e8-b176-44dd-a2ff-dea09229e268 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:32.547092 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 484/964] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:07:32 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:07:32.548189 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5684d259-d324-4cdc-a4c3-0263d9ab074e None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:32.548480 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5684d259-d324-4cdc-a4c3-0263d9ab074e None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:32.548761 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5684d259-d324-4cdc-a4c3-0263d9ab074e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:32.566724 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-5684d259-d324-4cdc-a4c3-0263d9ab074e None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:07:32.567961 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-5684d259-d324-4cdc-a4c3-0263d9ab074e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:07:32.575083 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baef360d-599b-4625-bdca-e5f2c13a7f19 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:32.592345 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:32.610618 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbc604a-4e94-48a1-9bbe-353ed209250f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:32.611384 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbc604a-4e94-48a1-9bbe-353ed209250f None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:32.611569 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbc604a-4e94-48a1-9bbe-353ed209250f None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:32.611871 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbc604a-4e94-48a1-9bbe-353ed209250f None admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:32.613249 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbc604a-4e94-48a1-9bbe-353ed209250f None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:07:32.613708 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbc604a-4e94-48a1-9bbe-353ed209250f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:32.614964 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbc604a-4e94-48a1-9bbe-353ed209250f None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:32.639334 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 481/965] 10.222.0.44 () {56 vars in 1149 bytes} [Wed Jun 29 11:07:32 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 11:07:32.639334 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbc604a-4e94-48a1-9bbe-353ed209250f None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:32.642260 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 485/966] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:07:32 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 29 11:07:32.642964 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5684d259-d324-4cdc-a4c3-0263d9ab074e None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:32.673234 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:32.744595 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5684d259-d324-4cdc-a4c3-0263d9ab074e None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:32.758207 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbc604a-4e94-48a1-9bbe-353ed209250f None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:32.781271 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70149d52-fd93-4fff-85a5-a458617669ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:32.781918 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70149d52-fd93-4fff-85a5-a458617669ae None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:32.782202 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70149d52-fd93-4fff-85a5-a458617669ae None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:32.782483 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70149d52-fd93-4fff-85a5-a458617669ae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:32.794358 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d2f6ce-c8d7-4ecb-9b25-c0f35f691fd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:32.795475 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d2f6ce-c8d7-4ecb-9b25-c0f35f691fd3 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:32.795767 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d2f6ce-c8d7-4ecb-9b25-c0f35f691fd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:32.796226 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d2f6ce-c8d7-4ecb-9b25-c0f35f691fd3 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:32.796880 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d2f6ce-c8d7-4ecb-9b25-c0f35f691fd3 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:32.798544 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d2f6ce-c8d7-4ecb-9b25-c0f35f691fd3 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:32.810780 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-43d2f6ce-c8d7-4ecb-9b25-c0f35f691fd3 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 29 11:07:32.810780 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:07:32.810780 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 29 11:07:32.810780 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 29 11:07:32.810780 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 29 11:07:32.810780 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 29 11:07:32.810780 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 29 11:07:32.810780 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 11:07:32.810780 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 29 11:07:32.810780 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:07:32.810780 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 29 11:07:32.810780 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:07:32.810780 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:07:32.810780 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:07:32.810780 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:07:32.810780 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:07:32.810780 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:07:32.811855 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:07:32.811855 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:07:32.811855 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:07:32.811855 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:07:32.811855 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:07:32.811855 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:07:32.811855 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 29 11:07:32.811855 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 29 11:07:32.811855 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 29 11:07:32.811855 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 29 11:07:32.811855 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:07:32.811855 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:07:32.811855 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:07:32.811855 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:07:32.811855 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:07:32.811855 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:07:32.812779 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:07:32.812779 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jun 29 11:07:32.812779 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:07:32.812779 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:07:32.812779 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:07:32.812779 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:07:32.812779 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:07:32.812779 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:07:32.812779 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:07:32.812779 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:07:32.812779 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 29 11:07:32.812779 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 11:07:32.812779 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 29 11:07:32.812779 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:07:32.812779 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 482/967] 10.222.0.44 () {56 vars in 1126 bytes} [Wed Jun 29 11:07:32 2022] GET /identity/v3/services/object-store => generated 93 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:07:32.813625 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d2f6ce-c8d7-4ecb-9b25-c0f35f691fd3 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:32.858172 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70149d52-fd93-4fff-85a5-a458617669ae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:32.859659 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70149d52-fd93-4fff-85a5-a458617669ae None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:32.861356 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:32.890672 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:32.936049 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d2f6ce-c8d7-4ecb-9b25-c0f35f691fd3 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:32.965183 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 486/968] 10.222.0.44 () {58 vars in 1282 bytes} [Wed Jun 29 11:07:32 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 29 11:07:32.966031 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70149d52-fd93-4fff-85a5-a458617669ae None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:32.979529 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7247ba9e-0c38-4e77-9e18-b840fb23b494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:32.980247 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7247ba9e-0c38-4e77-9e18-b840fb23b494 None admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:32.980536 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7247ba9e-0c38-4e77-9e18-b840fb23b494 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:32.980825 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7247ba9e-0c38-4e77-9e18-b840fb23b494 None admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:32.981513 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7247ba9e-0c38-4e77-9e18-b840fb23b494 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:32.982935 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7247ba9e-0c38-4e77-9e18-b840fb23b494 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:33.031366 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.sql.core [None req-7247ba9e-0c38-4e77-9e18-b840fb23b494 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 29 11:07:33.031366 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 29 11:07:33.031366 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [parameters: {'user_id': '3c498ab76be045829029cef53798a305', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 29 11:07:33.031366 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80166) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 29 11:07:33.041846 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-7247ba9e-0c38-4e77-9e18-b840fb23b494 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 29 11:07:33.041846 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:07:33.041846 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 29 11:07:33.041846 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 29 11:07:33.041846 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 29 11:07:33.041846 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 29 11:07:33.041846 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 29 11:07:33.041846 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application result = self._query(query) Jun 29 11:07:33.041846 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 29 11:07:33.041846 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application conn.query(q) Jun 29 11:07:33.041846 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 29 11:07:33.041846 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 29 11:07:33.041846 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 29 11:07:33.041846 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application result.read() Jun 29 11:07:33.041846 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 29 11:07:33.042764 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 29 11:07:33.042764 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 29 11:07:33.042764 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 29 11:07:33.042764 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 29 11:07:33.042764 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 29 11:07:33.042764 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 29 11:07:33.042764 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 29 11:07:33.042764 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 29 11:07:33.042764 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:07:33.042764 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 29 11:07:33.042764 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:07:33.042764 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:07:33.042764 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 29 11:07:33.042764 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 29 11:07:33.042764 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jun 29 11:07:33.042764 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jun 29 11:07:33.042764 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 29 11:07:33.042764 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application next(self.gen) Jun 29 11:07:33.049195 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 29 11:07:33.049195 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application yield resource Jun 29 11:07:33.049195 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 29 11:07:33.049195 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application next(self.gen) Jun 29 11:07:33.049195 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 29 11:07:33.049195 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self.session.rollback() Jun 29 11:07:33.049195 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 29 11:07:33.049195 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self.force_reraise() Jun 29 11:07:33.049195 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 29 11:07:33.049195 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise self.value Jun 29 11:07:33.049195 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 29 11:07:33.049195 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 29 11:07:33.049195 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 29 11:07:33.049195 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application session.commit() Jun 29 11:07:33.049195 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 29 11:07:33.049195 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 29 11:07:33.049195 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 29 11:07:33.050782 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self._prepare_impl() Jun 29 11:07:33.050782 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 29 11:07:33.050782 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self.session.flush() Jun 29 11:07:33.050782 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 29 11:07:33.050782 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self._flush(objects) Jun 29 11:07:33.050782 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 29 11:07:33.050782 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 29 11:07:33.050782 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 29 11:07:33.050782 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application compat.raise_( Jun 29 11:07:33.050782 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 29 11:07:33.050782 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception Jun 29 11:07:33.050782 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 29 11:07:33.050782 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application flush_context.execute() Jun 29 11:07:33.050782 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 29 11:07:33.050782 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rec.execute(self) Jun 29 11:07:33.050782 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 29 11:07:33.050782 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 29 11:07:33.051695 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 29 11:07:33.051695 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 29 11:07:33.051695 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jun 29 11:07:33.051695 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application result = connection._execute_20( Jun 29 11:07:33.051695 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 29 11:07:33.051695 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 29 11:07:33.051695 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jun 29 11:07:33.051695 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 29 11:07:33.051695 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 29 11:07:33.051695 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 29 11:07:33.051695 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 29 11:07:33.051695 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 29 11:07:33.051695 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 29 11:07:33.051695 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 29 11:07:33.051695 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 29 11:07:33.051695 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception Jun 29 11:07:33.054689 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 29 11:07:33.054689 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 29 11:07:33.054689 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 29 11:07:33.054689 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 29 11:07:33.054689 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 29 11:07:33.054689 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application result = self._query(query) Jun 29 11:07:33.054689 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 29 11:07:33.054689 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application conn.query(q) Jun 29 11:07:33.054689 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 29 11:07:33.054689 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 29 11:07:33.054689 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 29 11:07:33.054689 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application result.read() Jun 29 11:07:33.054689 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 29 11:07:33.054689 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 29 11:07:33.054689 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 29 11:07:33.054689 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 29 11:07:33.055714 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 29 11:07:33.055714 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 29 11:07:33.055714 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 29 11:07:33.055714 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 29 11:07:33.055714 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 29 11:07:33.055714 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 29 11:07:33.055714 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [parameters: {'user_id': '3c498ab76be045829029cef53798a305', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 29 11:07:33.055714 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 29 11:07:33.055714 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:07:33.055714 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 29 11:07:33.055714 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:07:33.055714 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:07:33.055714 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:07:33.055714 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:07:33.055714 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:07:33.056818 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:07:33.056818 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:07:33.056818 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:07:33.056818 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:07:33.056818 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:07:33.056818 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:07:33.056818 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:07:33.056818 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jun 29 11:07:33.056818 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jun 29 11:07:33.056818 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:07:33.056818 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:07:33.056818 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 11:07:33.056818 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 11:07:33.056818 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 11:07:33.056818 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 11:07:33.056818 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jun 29 11:07:33.056818 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jun 29 11:07:33.057987 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jun 29 11:07:33.057987 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jun 29 11:07:33.057987 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 29 11:07:33.057987 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 29 11:07:33.057987 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 29 11:07:33.057987 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:07:33.057987 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 483/969] 10.222.0.44 () {60 vars in 1131 bytes} [Wed Jun 29 11:07:32 2022] POST /identity/v3/users => generated 160 bytes in 231 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:07:33.057987 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7247ba9e-0c38-4e77-9e18-b840fb23b494 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:33.076240 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70149d52-fd93-4fff-85a5-a458617669ae None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:33.107546 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:33.113503 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e19ff5-abee-47fb-85a6-b907bd2e8cf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:33.114110 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e19ff5-abee-47fb-85a6-b907bd2e8cf2 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:33.114391 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e19ff5-abee-47fb-85a6-b907bd2e8cf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:33.114655 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e19ff5-abee-47fb-85a6-b907bd2e8cf2 None admin] PATH_INFO: `/v3/services` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:33.115358 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e19ff5-abee-47fb-85a6-b907bd2e8cf2 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:07:33.115659 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e19ff5-abee-47fb-85a6-b907bd2e8cf2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:33.117065 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e19ff5-abee-47fb-85a6-b907bd2e8cf2 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:33.121959 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 487/970] 10.222.0.44 () {56 vars in 1122 bytes} [Wed Jun 29 11:07:32 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:07:33.127018 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e19ff5-abee-47fb-85a6-b907bd2e8cf2 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:33.155514 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:33.159726 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7247ba9e-0c38-4e77-9e18-b840fb23b494 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:33.198338 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9415747a-c46d-4180-a4f0-8d37c9756d45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:33.199168 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9415747a-c46d-4180-a4f0-8d37c9756d45 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:33.199450 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9415747a-c46d-4180-a4f0-8d37c9756d45 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:33.199723 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9415747a-c46d-4180-a4f0-8d37c9756d45 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:33.211559 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9415747a-c46d-4180-a4f0-8d37c9756d45 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:33.213227 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9415747a-c46d-4180-a4f0-8d37c9756d45 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:33.231361 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-9415747a-c46d-4180-a4f0-8d37c9756d45 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 11:07:33.231361 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:07:33.231361 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:07:33.231361 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:07:33.231361 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:07:33.231361 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:07:33.231361 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:07:33.231361 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:07:33.231361 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:07:33.231361 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:07:33.231361 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:07:33.231361 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:07:33.231361 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 11:07:33.231361 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 11:07:33.231361 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 11:07:33.232406 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 11:07:33.232406 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:07:33.232406 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:07:33.232406 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:07:33.232406 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:07:33.232406 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:07:33.232406 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:07:33.232406 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:07:33.232406 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jun 29 11:07:33.232406 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:07:33.232406 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:07:33.232406 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:07:33.232406 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:07:33.232406 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:07:33.232406 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:07:33.232406 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:07:33.232406 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:07:33.233646 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 11:07:33.233646 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 11:07:33.233646 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 11:07:33.233646 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:07:33.234699 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e19ff5-abee-47fb-85a6-b907bd2e8cf2 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:33.237965 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 484/971] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jun 29 11:07:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:07:33.238812 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9415747a-c46d-4180-a4f0-8d37c9756d45 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:33.271910 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fd4df6-1a55-4d90-98f1-9f2fe4347321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:33.273069 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fd4df6-1a55-4d90-98f1-9f2fe4347321 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:33.273539 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fd4df6-1a55-4d90-98f1-9f2fe4347321 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:33.274018 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fd4df6-1a55-4d90-98f1-9f2fe4347321 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:33.281861 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fd4df6-1a55-4d90-98f1-9f2fe4347321 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:33.283435 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fd4df6-1a55-4d90-98f1-9f2fe4347321 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:33.298245 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-b7fd4df6-1a55-4d90-98f1-9f2fe4347321 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 29 11:07:33.298245 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:07:33.298245 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:07:33.298245 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:07:33.298245 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:07:33.298245 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:07:33.298245 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:07:33.298245 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:07:33.298245 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:07:33.298245 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:07:33.298245 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:07:33.298245 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:07:33.298245 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 11:07:33.298245 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 11:07:33.298245 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 11:07:33.298245 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 11:07:33.299236 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:07:33.299236 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:07:33.299236 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 11:07:33.299236 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 11:07:33.299236 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 11:07:33.299236 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 11:07:33.299236 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:07:33.299236 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:07:33.299236 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:07:33.299236 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:07:33.299236 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:07:33.299236 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jun 29 11:07:33.299236 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:07:33.299236 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:07:33.299236 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:07:33.299236 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:07:33.299236 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:07:33.299236 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:07:33.305890 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:07:33.305890 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:07:33.305890 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 11:07:33.305890 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 11:07:33.305890 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 11:07:33.305890 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 11:07:33.305890 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 11:07:33.305890 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 11:07:33.305890 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 29 11:07:33.305890 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jun 29 11:07:33.305890 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 488/972] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Jun 29 11:07:33 2022] GET /identity/v3/users/cinder => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:07:33.305890 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fd4df6-1a55-4d90-98f1-9f2fe4347321 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:33.354532 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9415747a-c46d-4180-a4f0-8d37c9756d45 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:33.369131 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:33.396190 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba3e184-3307-4e39-bd7a-a3d65e830fd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:33.396856 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba3e184-3307-4e39-bd7a-a3d65e830fd7 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:33.397139 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba3e184-3307-4e39-bd7a-a3d65e830fd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:33.397415 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba3e184-3307-4e39-bd7a-a3d65e830fd7 None admin] PATH_INFO: `/v3/services` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:33.398120 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba3e184-3307-4e39-bd7a-a3d65e830fd7 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:07:33.398430 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba3e184-3307-4e39-bd7a-a3d65e830fd7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:33.399861 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba3e184-3307-4e39-bd7a-a3d65e830fd7 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:33.407688 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 485/973] 10.222.0.44 () {56 vars in 1122 bytes} [Wed Jun 29 11:07:33 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:07:33.408507 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba3e184-3307-4e39-bd7a-a3d65e830fd7 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:33.423621 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fd4df6-1a55-4d90-98f1-9f2fe4347321 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:33.430324 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:33.459539 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0570bffd-6db2-42f0-bdbe-f73dd6804405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:33.460230 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0570bffd-6db2-42f0-bdbe-f73dd6804405 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:33.460509 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0570bffd-6db2-42f0-bdbe-f73dd6804405 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:33.460782 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0570bffd-6db2-42f0-bdbe-f73dd6804405 None admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:33.461487 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0570bffd-6db2-42f0-bdbe-f73dd6804405 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:07:33.461790 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0570bffd-6db2-42f0-bdbe-f73dd6804405 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:33.463151 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0570bffd-6db2-42f0-bdbe-f73dd6804405 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:33.474132 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 489/974] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Jun 29 11:07:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:07:33.474957 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0570bffd-6db2-42f0-bdbe-f73dd6804405 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:33.512440 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba3e184-3307-4e39-bd7a-a3d65e830fd7 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:33.544420 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696750bf-3df3-485c-8d0e-eb78345f9324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:33.545082 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696750bf-3df3-485c-8d0e-eb78345f9324 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:33.545366 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696750bf-3df3-485c-8d0e-eb78345f9324 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:33.545640 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696750bf-3df3-485c-8d0e-eb78345f9324 None admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:33.546431 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696750bf-3df3-485c-8d0e-eb78345f9324 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:07:33.546739 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696750bf-3df3-485c-8d0e-eb78345f9324 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:33.548176 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696750bf-3df3-485c-8d0e-eb78345f9324 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:33.572535 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0570bffd-6db2-42f0-bdbe-f73dd6804405 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:33.583085 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 486/975] 10.222.0.44 () {56 vars in 1101 bytes} [Wed Jun 29 11:07:33 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:07:33.585850 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f895bd-d2a6-41ee-8e9a-ee1ecfbc97fe None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:33.585850 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f895bd-d2a6-41ee-8e9a-ee1ecfbc97fe None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:33.585850 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f895bd-d2a6-41ee-8e9a-ee1ecfbc97fe None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:33.587589 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 487/976] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:07:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:07:33.587589 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f895bd-d2a6-41ee-8e9a-ee1ecfbc97fe None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:33.593825 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:33.616345 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03a3086-1a66-4dbd-bb48-9b889e9fcb40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:33.617027 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03a3086-1a66-4dbd-bb48-9b889e9fcb40 None admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:33.617307 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03a3086-1a66-4dbd-bb48-9b889e9fcb40 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:33.625047 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03a3086-1a66-4dbd-bb48-9b889e9fcb40 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:33.625766 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03a3086-1a66-4dbd-bb48-9b889e9fcb40 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:33.627178 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03a3086-1a66-4dbd-bb48-9b889e9fcb40 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:33.682000 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:33.710172 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 490/977] 10.222.0.44 () {60 vars in 1144 bytes} [Wed Jun 29 11:07:33 2022] POST /identity/v3/endpoints => generated 349 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:07:33.712265 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c619d13-7ab3-4be4-a46b-1f8868b762b3 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:33.712752 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c619d13-7ab3-4be4-a46b-1f8868b762b3 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:33.713251 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c619d13-7ab3-4be4-a46b-1f8868b762b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:33.725674 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f895bd-d2a6-41ee-8e9a-ee1ecfbc97fe None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:33.746982 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-8c619d13-7ab3-4be4-a46b-1f8868b762b3 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:07:33.749580 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-8c619d13-7ab3-4be4-a46b-1f8868b762b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:07:33.781347 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55399b7-2abb-4fc5-ae14-e160ed31195a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:33.782185 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55399b7-2abb-4fc5-ae14-e160ed31195a None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:33.782466 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55399b7-2abb-4fc5-ae14-e160ed31195a None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:33.782734 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55399b7-2abb-4fc5-ae14-e160ed31195a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:33.790591 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55399b7-2abb-4fc5-ae14-e160ed31195a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:33.792101 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55399b7-2abb-4fc5-ae14-e160ed31195a None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:33.809569 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-e55399b7-2abb-4fc5-ae14-e160ed31195a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 11:07:33.809569 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:07:33.809569 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:07:33.809569 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:07:33.809569 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:07:33.809569 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:07:33.809569 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:07:33.809569 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:07:33.809569 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:07:33.809569 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:07:33.809569 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:07:33.809569 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:07:33.809569 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 11:07:33.809569 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 11:07:33.809569 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 11:07:33.810607 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 11:07:33.810607 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:07:33.810607 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:07:33.810607 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:07:33.810607 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:07:33.810607 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:07:33.810607 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:07:33.810607 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:07:33.810607 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jun 29 11:07:33.810607 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:07:33.810607 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:07:33.810607 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:07:33.810607 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:07:33.810607 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:07:33.810607 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:07:33.810607 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:07:33.810607 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:07:33.811528 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 11:07:33.811528 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 11:07:33.811528 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 11:07:33.811528 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 11:07:33.811528 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 11:07:33.811528 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 11:07:33.811528 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 11:07:33.811528 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:07:33.811528 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 488/978] 10.222.0.44 () {56 vars in 1111 bytes} [Wed Jun 29 11:07:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:07:33.817402 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55399b7-2abb-4fc5-ae14-e160ed31195a None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:33.839746 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 491/979] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:07:33 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 11:07:33.864486 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c619d13-7ab3-4be4-a46b-1f8868b762b3 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:33.942416 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55399b7-2abb-4fc5-ae14-e160ed31195a None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:33.984903 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c619d13-7ab3-4be4-a46b-1f8868b762b3 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:33.985715 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:33.986088 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:33.992994 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4e22c1-7d0e-4c7e-990c-c14b3cc206bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:33.993658 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4e22c1-7d0e-4c7e-990c-c14b3cc206bf None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:33.993941 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4e22c1-7d0e-4c7e-990c-c14b3cc206bf None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:33.994386 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4e22c1-7d0e-4c7e-990c-c14b3cc206bf None admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:33.995328 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4e22c1-7d0e-4c7e-990c-c14b3cc206bf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:07:33.995328 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4e22c1-7d0e-4c7e-990c-c14b3cc206bf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:33.996775 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4e22c1-7d0e-4c7e-990c-c14b3cc206bf None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:34.014047 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 489/980] 10.222.0.44 () {56 vars in 1148 bytes} [Wed Jun 29 11:07:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 11:07:34.015513 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4e22c1-7d0e-4c7e-990c-c14b3cc206bf None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:34.028828 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21887a01-b0e0-4efe-8677-3b0621dba006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:34.029496 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21887a01-b0e0-4efe-8677-3b0621dba006 None admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:34.029778 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21887a01-b0e0-4efe-8677-3b0621dba006 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:34.030052 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21887a01-b0e0-4efe-8677-3b0621dba006 None admin] PATH_INFO: `/v3/services` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:34.030773 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21887a01-b0e0-4efe-8677-3b0621dba006 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:34.032174 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21887a01-b0e0-4efe-8677-3b0621dba006 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:34.113649 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4e22c1-7d0e-4c7e-990c-c14b3cc206bf None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:34.140089 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 492/981] 10.222.0.44 () {60 vars in 1141 bytes} [Wed Jun 29 11:07:33 2022] POST /identity/v3/services => generated 242 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 11:07:34.161004 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487992f9-493d-4287-882a-f18d7c40321c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:34.161678 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487992f9-493d-4287-882a-f18d7c40321c None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:34.161964 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487992f9-493d-4287-882a-f18d7c40321c None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:34.162247 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487992f9-493d-4287-882a-f18d7c40321c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:34.163090 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487992f9-493d-4287-882a-f18d7c40321c None admin] RBAC: Adding query filter params (role.id=3e8ef91b147f4b6b803ef07f3301be3f, scope.project.id=ebe5dffa4bb8440286687daefb6c98f4, user.id=55487c2107a44c15b656b4865ee0d4fa) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:07:34.163465 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487992f9-493d-4287-882a-f18d7c40321c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:34.165070 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487992f9-493d-4287-882a-f18d7c40321c None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:34.179958 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 490/982] 10.222.0.44 () {56 vars in 1374 bytes} [Wed Jun 29 11:07:34 2022] GET /identity/v3/role_assignments?user.id=55487c2107a44c15b656b4865ee0d4fa&scope.project.id=ebe5dffa4bb8440286687daefb6c98f4&role.id=3e8ef91b147f4b6b803ef07f3301be3f => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:07:35.822652 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70916ae8-60e3-4221-8270-94128cf40f1e None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:35.823195 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70916ae8-60e3-4221-8270-94128cf40f1e None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:35.823195 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70916ae8-60e3-4221-8270-94128cf40f1e None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:35.824580 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 493/983] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:07:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:07:35.829769 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790d8057-b50c-4185-b3a4-f7347142c2a0 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:35.830145 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790d8057-b50c-4185-b3a4-f7347142c2a0 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:35.830311 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790d8057-b50c-4185-b3a4-f7347142c2a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:35.848083 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-790d8057-b50c-4185-b3a4-f7347142c2a0 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:07:35.849584 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-790d8057-b50c-4185-b3a4-f7347142c2a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:07:35.897950 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:35.928137 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 491/984] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:07:35 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 11:07:35.947972 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70916ae8-60e3-4221-8270-94128cf40f1e None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:36.050149 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70916ae8-60e3-4221-8270-94128cf40f1e None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:36.084452 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5c36a6-1b38-4a8a-9c08-1acfd29d5502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:36.085217 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5c36a6-1b38-4a8a-9c08-1acfd29d5502 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:36.085501 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5c36a6-1b38-4a8a-9c08-1acfd29d5502 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:36.085773 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5c36a6-1b38-4a8a-9c08-1acfd29d5502 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:36.086417 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5c36a6-1b38-4a8a-9c08-1acfd29d5502 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:36.087776 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5c36a6-1b38-4a8a-9c08-1acfd29d5502 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:36.099767 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-ef5c36a6-1b38-4a8a-9c08-1acfd29d5502 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 29 11:07:36.099767 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:07:36.099767 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 29 11:07:36.099767 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 29 11:07:36.099767 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 29 11:07:36.099767 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 29 11:07:36.099767 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 29 11:07:36.099767 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 11:07:36.099767 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 29 11:07:36.099767 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jun 29 11:07:36.099767 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 29 11:07:36.099767 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jun 29 11:07:36.099767 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:07:36.099767 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:07:36.099767 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:07:36.099767 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:07:36.099767 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:07:36.100904 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:07:36.100904 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:07:36.100904 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:07:36.100904 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:07:36.100904 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:07:36.100904 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:07:36.100904 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 29 11:07:36.100904 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 29 11:07:36.100904 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 29 11:07:36.100904 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 29 11:07:36.100904 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:07:36.100904 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:07:36.100904 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:07:36.100904 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:07:36.100904 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:07:36.100904 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:07:36.101753 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:07:36.101753 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jun 29 11:07:36.101753 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:07:36.101753 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:07:36.101753 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:07:36.101753 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:07:36.101753 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:07:36.101753 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:07:36.101753 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:07:36.101753 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:07:36.101753 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 29 11:07:36.101753 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 11:07:36.101753 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 29 11:07:36.101753 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jun 29 11:07:36.104148 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 494/985] 10.222.0.44 () {56 vars in 1126 bytes} [Wed Jun 29 11:07:35 2022] GET /identity/v3/services/object-store => generated 93 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:07:36.106865 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790d8057-b50c-4185-b3a4-f7347142c2a0 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:36.205645 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790d8057-b50c-4185-b3a4-f7347142c2a0 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:36.208989 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e61bfc-b793-40df-8394-efc335841d9a None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:36.209272 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e61bfc-b793-40df-8394-efc335841d9a None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:36.209559 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e61bfc-b793-40df-8394-efc335841d9a None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:36.210928 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 495/986] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:07:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 29 11:07:36.216210 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3635a33d-6038-4aa1-8f9d-5d7811989663 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:36.216503 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3635a33d-6038-4aa1-8f9d-5d7811989663 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:36.216779 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3635a33d-6038-4aa1-8f9d-5d7811989663 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:36.233714 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-3635a33d-6038-4aa1-8f9d-5d7811989663 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:07:36.235000 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-3635a33d-6038-4aa1-8f9d-5d7811989663 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:07:36.242592 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b405976e-807f-40f4-8e45-9eb069e1be74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:36.243441 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b405976e-807f-40f4-8e45-9eb069e1be74 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:36.243896 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b405976e-807f-40f4-8e45-9eb069e1be74 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:36.244339 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b405976e-807f-40f4-8e45-9eb069e1be74 None admin] PATH_INFO: `/v3/services` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:36.245178 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b405976e-807f-40f4-8e45-9eb069e1be74 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:07:36.245624 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b405976e-807f-40f4-8e45-9eb069e1be74 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:36.247224 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b405976e-807f-40f4-8e45-9eb069e1be74 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:36.252138 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 492/987] 10.222.0.44 () {56 vars in 1122 bytes} [Wed Jun 29 11:07:36 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:07:36.253938 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7028c8ef-58d5-4c4d-886c-5d330e6dab8c None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:36.254220 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7028c8ef-58d5-4c4d-886c-5d330e6dab8c None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:36.254490 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7028c8ef-58d5-4c4d-886c-5d330e6dab8c None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:36.255701 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 493/988] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:07:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:07:36.258084 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7028c8ef-58d5-4c4d-886c-5d330e6dab8c None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:36.262961 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:36.304538 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:36.308833 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 496/989] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:07:36 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 29 11:07:36.310223 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ffb3cc-cb0d-4b63-b664-8aa5f40710bf None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:36.310861 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ffb3cc-cb0d-4b63-b664-8aa5f40710bf None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:36.311447 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ffb3cc-cb0d-4b63-b664-8aa5f40710bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:36.331254 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-36ffb3cc-cb0d-4b63-b664-8aa5f40710bf None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:07:36.332541 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-36ffb3cc-cb0d-4b63-b664-8aa5f40710bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:07:36.362630 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7028c8ef-58d5-4c4d-886c-5d330e6dab8c None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:36.395604 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0feae465-1851-401c-a7aa-64aa4aed9f94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:36.395957 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0feae465-1851-401c-a7aa-64aa4aed9f94 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:36.396238 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0feae465-1851-401c-a7aa-64aa4aed9f94 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:36.396515 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0feae465-1851-401c-a7aa-64aa4aed9f94 None admin] PATH_INFO: `/v3/services` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:36.397383 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0feae465-1851-401c-a7aa-64aa4aed9f94 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:07:36.397518 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0feae465-1851-401c-a7aa-64aa4aed9f94 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:36.398949 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0feae465-1851-401c-a7aa-64aa4aed9f94 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:36.403300 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 497/990] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:07:36 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 11:07:36.409093 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 494/991] 10.222.0.44 () {56 vars in 1122 bytes} [Wed Jun 29 11:07:36 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 11:07:36.409536 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ffb3cc-cb0d-4b63-b664-8aa5f40710bf None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:36.410216 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0feae465-1851-401c-a7aa-64aa4aed9f94 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:36.495050 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:36.509812 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ffb3cc-cb0d-4b63-b664-8aa5f40710bf None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:36.514571 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0feae465-1851-401c-a7aa-64aa4aed9f94 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:36.525667 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:36.542487 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b444bc0f-7cdc-470d-afe9-1c448c586fa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:36.543327 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b444bc0f-7cdc-470d-afe9-1c448c586fa0 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:36.543611 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b444bc0f-7cdc-470d-afe9-1c448c586fa0 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:36.544902 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b444bc0f-7cdc-470d-afe9-1c448c586fa0 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:36.544902 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b444bc0f-7cdc-470d-afe9-1c448c586fa0 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:36.546066 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b444bc0f-7cdc-470d-afe9-1c448c586fa0 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:36.550129 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7140ec74-75b7-4226-96ba-99ebb0efb47a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:36.550782 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7140ec74-75b7-4226-96ba-99ebb0efb47a None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:36.551060 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7140ec74-75b7-4226-96ba-99ebb0efb47a None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:36.551331 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7140ec74-75b7-4226-96ba-99ebb0efb47a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:36.552158 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7140ec74-75b7-4226-96ba-99ebb0efb47a None admin] RBAC: Adding query filter params (interface=admin, service_id=99486dc5d2414777a654e34cc11881c2, region_id=RegionOne) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:07:36.552473 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7140ec74-75b7-4226-96ba-99ebb0efb47a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:36.553819 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7140ec74-75b7-4226-96ba-99ebb0efb47a None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:36.558414 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-b444bc0f-7cdc-470d-afe9-1c448c586fa0 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 29 11:07:36.558414 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:07:36.558414 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 29 11:07:36.558414 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 29 11:07:36.558414 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 29 11:07:36.558414 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 29 11:07:36.558414 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 29 11:07:36.558414 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 11:07:36.558414 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 29 11:07:36.558414 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jun 29 11:07:36.558414 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 29 11:07:36.558414 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jun 29 11:07:36.558414 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:07:36.558414 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:07:36.558414 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:07:36.558414 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:07:36.558414 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:07:36.559431 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:07:36.559431 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:07:36.559431 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:07:36.559431 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:07:36.559431 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:07:36.559431 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:07:36.559431 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 29 11:07:36.559431 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 29 11:07:36.559431 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 29 11:07:36.559431 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 29 11:07:36.559431 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:07:36.559431 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:07:36.559431 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:07:36.559431 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:07:36.559431 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:07:36.559431 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:07:36.560337 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:07:36.560337 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jun 29 11:07:36.560337 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:07:36.560337 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:07:36.560337 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:07:36.560337 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:07:36.560337 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:07:36.560337 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:07:36.560337 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:07:36.560337 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:07:36.560337 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 29 11:07:36.560337 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 11:07:36.560337 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 29 11:07:36.560337 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jun 29 11:07:36.561080 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 495/992] 10.222.0.44 () {56 vars in 1249 bytes} [Wed Jun 29 11:07:36 2022] GET /identity/v3/endpoints?service_id=99486dc5d2414777a654e34cc11881c2&interface=admin®ion_id=RegionOne => generated 193 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:07:36.561162 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 498/993] 10.222.0.44 () {56 vars in 1111 bytes} [Wed Jun 29 11:07:36 2022] GET /identity/v3/services/compute => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:07:36.561242 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7140ec74-75b7-4226-96ba-99ebb0efb47a None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:36.565195 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b444bc0f-7cdc-470d-afe9-1c448c586fa0 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:36.662000 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b444bc0f-7cdc-470d-afe9-1c448c586fa0 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:36.662180 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7140ec74-75b7-4226-96ba-99ebb0efb47a None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:36.694788 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b400130-67e7-4c87-b949-771cefc8fc3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:36.695439 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b400130-67e7-4c87-b949-771cefc8fc3f None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:36.695720 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b400130-67e7-4c87-b949-771cefc8fc3f None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:36.696028 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b400130-67e7-4c87-b949-771cefc8fc3f None admin] PATH_INFO: `/v3/services` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:36.696838 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7a86dc-bd51-460c-8cff-81813a1dc5f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:36.696962 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b400130-67e7-4c87-b949-771cefc8fc3f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:36.697610 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7a86dc-bd51-460c-8cff-81813a1dc5f6 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:36.697891 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7a86dc-bd51-460c-8cff-81813a1dc5f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:36.698169 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7a86dc-bd51-460c-8cff-81813a1dc5f6 None admin] PATH_INFO: `/v3/services/image` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:36.698298 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b400130-67e7-4c87-b949-771cefc8fc3f None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:36.698804 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7a86dc-bd51-460c-8cff-81813a1dc5f6 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:36.700180 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7a86dc-bd51-460c-8cff-81813a1dc5f6 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:36.704520 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 499/994] 10.222.0.44 () {56 vars in 1087 bytes} [Wed Jun 29 11:07:36 2022] GET /identity/v3/services => generated 1727 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 11:07:36.705097 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b400130-67e7-4c87-b949-771cefc8fc3f None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:36.712001 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-ce7a86dc-bd51-460c-8cff-81813a1dc5f6 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 29 11:07:36.712001 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:07:36.712001 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 29 11:07:36.712001 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 29 11:07:36.712001 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 29 11:07:36.712001 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 29 11:07:36.712001 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 29 11:07:36.712001 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 11:07:36.712001 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 29 11:07:36.712001 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:07:36.712001 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 29 11:07:36.712001 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:07:36.712001 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:07:36.712001 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:07:36.712001 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:07:36.712001 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:07:36.712001 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:07:36.713062 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:07:36.713062 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:07:36.713062 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:07:36.713062 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:07:36.713062 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:07:36.713062 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:07:36.713062 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 29 11:07:36.713062 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 29 11:07:36.713062 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 29 11:07:36.713062 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 29 11:07:36.713062 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:07:36.713062 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:07:36.713062 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:07:36.713062 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:07:36.713062 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:07:36.713062 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:07:36.713999 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:07:36.713999 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jun 29 11:07:36.713999 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:07:36.713999 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:07:36.713999 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:07:36.713999 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:07:36.713999 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:07:36.713999 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:07:36.713999 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:07:36.713999 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:07:36.713999 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 29 11:07:36.713999 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 11:07:36.713999 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 29 11:07:36.713999 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:07:36.713999 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 496/995] 10.222.0.44 () {56 vars in 1105 bytes} [Wed Jun 29 11:07:36 2022] GET /identity/v3/services/image => generated 86 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:07:36.719400 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7a86dc-bd51-460c-8cff-81813a1dc5f6 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:36.734133 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:36.761413 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:36.827559 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b400130-67e7-4c87-b949-771cefc8fc3f None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:36.837314 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7a86dc-bd51-460c-8cff-81813a1dc5f6 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:36.864896 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ca60e3-3fc8-48fb-9ba7-5c8413e7513a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:36.865565 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ca60e3-3fc8-48fb-9ba7-5c8413e7513a None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:36.865847 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ca60e3-3fc8-48fb-9ba7-5c8413e7513a None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:36.866124 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ca60e3-3fc8-48fb-9ba7-5c8413e7513a None admin] PATH_INFO: `/v3/services` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:36.866835 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ca60e3-3fc8-48fb-9ba7-5c8413e7513a None admin] RBAC: Adding query filter params (name=compute) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:07:36.867147 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ca60e3-3fc8-48fb-9ba7-5c8413e7513a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:36.868541 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ca60e3-3fc8-48fb-9ba7-5c8413e7513a None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:36.876562 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 500/996] 10.222.0.44 () {56 vars in 1112 bytes} [Wed Jun 29 11:07:36 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:07:36.879768 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96edefdf-0c84-408e-9d6b-39d397e67a1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:36.880554 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96edefdf-0c84-408e-9d6b-39d397e67a1c None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:36.881492 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96edefdf-0c84-408e-9d6b-39d397e67a1c None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:36.881492 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96edefdf-0c84-408e-9d6b-39d397e67a1c None admin] PATH_INFO: `/v3/services` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:36.881660 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ca60e3-3fc8-48fb-9ba7-5c8413e7513a None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:36.881862 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96edefdf-0c84-408e-9d6b-39d397e67a1c None admin] RBAC: Adding query filter params (name=image) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:07:36.882466 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96edefdf-0c84-408e-9d6b-39d397e67a1c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:36.883455 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96edefdf-0c84-408e-9d6b-39d397e67a1c None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:36.891472 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 497/997] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jun 29 11:07:36 2022] GET /identity/v3/services?name=image => generated 122 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:07:36.897764 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96edefdf-0c84-408e-9d6b-39d397e67a1c None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:36.968149 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:36.997129 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ca60e3-3fc8-48fb-9ba7-5c8413e7513a None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:37.005471 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:37.009993 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96edefdf-0c84-408e-9d6b-39d397e67a1c None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:37.032502 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8831d62f-4145-4f0d-b7c2-6a609fddc4bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:37.033159 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8831d62f-4145-4f0d-b7c2-6a609fddc4bf None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:37.033438 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8831d62f-4145-4f0d-b7c2-6a609fddc4bf None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:37.033852 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8831d62f-4145-4f0d-b7c2-6a609fddc4bf None admin] PATH_INFO: `/v3/services` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:37.035179 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8831d62f-4145-4f0d-b7c2-6a609fddc4bf None admin] RBAC: Adding query filter params (type=compute) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:07:37.035485 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8831d62f-4145-4f0d-b7c2-6a609fddc4bf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:37.036877 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8831d62f-4145-4f0d-b7c2-6a609fddc4bf None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:37.042350 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 501/998] 10.222.0.44 () {56 vars in 1112 bytes} [Wed Jun 29 11:07:36 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 11:07:37.047386 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d58423c-aa39-49cd-8cb9-d31b63af8806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:37.048014 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8831d62f-4145-4f0d-b7c2-6a609fddc4bf None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:37.048630 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d58423c-aa39-49cd-8cb9-d31b63af8806 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:37.049140 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d58423c-aa39-49cd-8cb9-d31b63af8806 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:37.049797 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d58423c-aa39-49cd-8cb9-d31b63af8806 None admin] PATH_INFO: `/v3/services` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:37.051031 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d58423c-aa39-49cd-8cb9-d31b63af8806 None admin] RBAC: Adding query filter params (type=image) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:07:37.052574 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d58423c-aa39-49cd-8cb9-d31b63af8806 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:37.054171 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d58423c-aa39-49cd-8cb9-d31b63af8806 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:37.061778 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 498/999] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jun 29 11:07:36 2022] GET /identity/v3/services?type=image => generated 122 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 11:07:37.164708 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8831d62f-4145-4f0d-b7c2-6a609fddc4bf None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:37.205086 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a86992-03ce-41ad-9e8f-aa3a5acaae76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:37.206574 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a86992-03ce-41ad-9e8f-aa3a5acaae76 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:37.206574 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a86992-03ce-41ad-9e8f-aa3a5acaae76 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:37.206850 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a86992-03ce-41ad-9e8f-aa3a5acaae76 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:37.207617 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a86992-03ce-41ad-9e8f-aa3a5acaae76 None admin] RBAC: Adding query filter params (interface=public, service_id=b253f886edf840c1a52617dae2c252de, region_id=RegionOne) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:07:37.207949 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a86992-03ce-41ad-9e8f-aa3a5acaae76 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:37.209323 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a86992-03ce-41ad-9e8f-aa3a5acaae76 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:37.215456 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 502/1000] 10.222.0.44 () {56 vars in 1251 bytes} [Wed Jun 29 11:07:37 2022] GET /identity/v3/endpoints?service_id=b253f886edf840c1a52617dae2c252de&interface=public®ion_id=RegionOne => generated 194 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 11:07:37.220837 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d58423c-aa39-49cd-8cb9-d31b63af8806 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:37.323600 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d58423c-aa39-49cd-8cb9-d31b63af8806 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:37.359915 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072d3db3-9401-46cb-a963-325fd2ed0636 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:37.360943 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072d3db3-9401-46cb-a963-325fd2ed0636 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:37.361437 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072d3db3-9401-46cb-a963-325fd2ed0636 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:37.361943 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072d3db3-9401-46cb-a963-325fd2ed0636 None admin] PATH_INFO: `/v3/services` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:37.362851 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072d3db3-9401-46cb-a963-325fd2ed0636 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:37.364467 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072d3db3-9401-46cb-a963-325fd2ed0636 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:37.371612 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 499/1001] 10.222.0.44 () {56 vars in 1087 bytes} [Wed Jun 29 11:07:37 2022] GET /identity/v3/services => generated 1727 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 11:07:38.777414 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab5803f-499c-4169-b736-f1afbf078f16 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:38.777955 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab5803f-499c-4169-b736-f1afbf078f16 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:38.777955 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab5803f-499c-4169-b736-f1afbf078f16 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:38.779463 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 503/1002] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:07:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:07:38.785014 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc62dbc-8521-42be-8719-da8ac4709072 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:38.785396 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc62dbc-8521-42be-8719-da8ac4709072 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:38.785585 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc62dbc-8521-42be-8719-da8ac4709072 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:38.801941 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-ffc62dbc-8521-42be-8719-da8ac4709072 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:07:38.803130 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-ffc62dbc-8521-42be-8719-da8ac4709072 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:07:38.826635 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:38.870825 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 500/1003] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:07:38 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 11:07:38.894587 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab5803f-499c-4169-b736-f1afbf078f16 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:38.903785 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:39.009873 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab5803f-499c-4169-b736-f1afbf078f16 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:39.051872 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2261de81-99f4-4faa-a613-85687a174c7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:39.052601 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2261de81-99f4-4faa-a613-85687a174c7a None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:39.052883 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2261de81-99f4-4faa-a613-85687a174c7a None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:39.053159 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2261de81-99f4-4faa-a613-85687a174c7a None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:39.053811 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2261de81-99f4-4faa-a613-85687a174c7a None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:39.055174 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2261de81-99f4-4faa-a613-85687a174c7a None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:39.067908 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-2261de81-99f4-4faa-a613-85687a174c7a None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 29 11:07:39.067908 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:07:39.067908 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 29 11:07:39.067908 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 29 11:07:39.067908 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 29 11:07:39.067908 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 29 11:07:39.067908 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 29 11:07:39.067908 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 11:07:39.067908 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 29 11:07:39.067908 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jun 29 11:07:39.067908 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 29 11:07:39.067908 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jun 29 11:07:39.067908 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:07:39.067908 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:07:39.067908 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:07:39.067908 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:07:39.067908 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:07:39.068904 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:07:39.068904 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:07:39.068904 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:07:39.068904 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:07:39.068904 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:07:39.068904 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:07:39.068904 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 29 11:07:39.068904 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 29 11:07:39.068904 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 29 11:07:39.068904 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 29 11:07:39.068904 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:07:39.068904 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:07:39.068904 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:07:39.068904 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:07:39.068904 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:07:39.068904 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:07:39.069768 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:07:39.069768 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jun 29 11:07:39.069768 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:07:39.069768 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:07:39.069768 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:07:39.069768 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:07:39.069768 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:07:39.069768 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:07:39.069768 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:07:39.069768 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:07:39.069768 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 29 11:07:39.069768 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 11:07:39.069768 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 29 11:07:39.069768 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jun 29 11:07:39.072194 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 504/1004] 10.222.0.44 () {56 vars in 1126 bytes} [Wed Jun 29 11:07:38 2022] GET /identity/v3/services/object-store => generated 93 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:07:39.075258 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc62dbc-8521-42be-8719-da8ac4709072 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:39.086904 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42a1e6c-4799-4aac-a533-52df7b48cc84 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:39.087237 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42a1e6c-4799-4aac-a533-52df7b48cc84 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:39.087469 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42a1e6c-4799-4aac-a533-52df7b48cc84 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:39.089098 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 505/1005] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:07:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 29 11:07:39.093744 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9acb7c3-534e-4222-bc4a-95d31a9335bf None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:39.094028 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9acb7c3-534e-4222-bc4a-95d31a9335bf None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:39.094306 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9acb7c3-534e-4222-bc4a-95d31a9335bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:39.110997 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-a9acb7c3-534e-4222-bc4a-95d31a9335bf None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:07:39.112245 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-a9acb7c3-534e-4222-bc4a-95d31a9335bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:07:39.180215 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc62dbc-8521-42be-8719-da8ac4709072 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:39.185480 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:39.190771 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 506/1006] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:07:39 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 29 11:07:39.214374 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9acb7c3-534e-4222-bc4a-95d31a9335bf None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:39.219054 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d06362d-9617-4c41-9de2-02ea790353a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:39.219970 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d06362d-9617-4c41-9de2-02ea790353a6 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:39.220254 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d06362d-9617-4c41-9de2-02ea790353a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:39.220526 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d06362d-9617-4c41-9de2-02ea790353a6 None admin] PATH_INFO: `/v3/services` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:39.221221 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d06362d-9617-4c41-9de2-02ea790353a6 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:07:39.221525 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d06362d-9617-4c41-9de2-02ea790353a6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:39.222876 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d06362d-9617-4c41-9de2-02ea790353a6 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:39.232779 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 501/1007] 10.222.0.44 () {56 vars in 1122 bytes} [Wed Jun 29 11:07:39 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:07:39.233928 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d06362d-9617-4c41-9de2-02ea790353a6 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:39.260303 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:39.325654 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9acb7c3-534e-4222-bc4a-95d31a9335bf None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:39.350750 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d06362d-9617-4c41-9de2-02ea790353a6 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:39.373924 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47110df7-286e-4d24-8d0d-248cd8e38bab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:39.374603 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47110df7-286e-4d24-8d0d-248cd8e38bab None admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:39.378725 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47110df7-286e-4d24-8d0d-248cd8e38bab None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:39.378725 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47110df7-286e-4d24-8d0d-248cd8e38bab None admin] PATH_INFO: `/v3/services` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:39.378725 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47110df7-286e-4d24-8d0d-248cd8e38bab None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:39.378725 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47110df7-286e-4d24-8d0d-248cd8e38bab None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:39.398298 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be717fe-5e27-4c9f-b013-33384d1e06ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:39.398966 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be717fe-5e27-4c9f-b013-33384d1e06ab None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:39.399247 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be717fe-5e27-4c9f-b013-33384d1e06ab None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:39.399518 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be717fe-5e27-4c9f-b013-33384d1e06ab None admin] PATH_INFO: `/v3/services` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:39.400258 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be717fe-5e27-4c9f-b013-33384d1e06ab None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:07:39.400561 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be717fe-5e27-4c9f-b013-33384d1e06ab None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:39.401911 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be717fe-5e27-4c9f-b013-33384d1e06ab None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:39.411283 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 502/1008] 10.222.0.44 () {56 vars in 1122 bytes} [Wed Jun 29 11:07:39 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:07:39.412550 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063b475e-6312-456b-b841-1a59f47fd1ef None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:39.412845 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063b475e-6312-456b-b841-1a59f47fd1ef None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:39.413129 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063b475e-6312-456b-b841-1a59f47fd1ef None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:39.414417 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 503/1009] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:07:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 29 11:07:39.415103 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063b475e-6312-456b-b841-1a59f47fd1ef None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:39.447548 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 507/1010] 10.222.0.44 () {60 vars in 1141 bytes} [Wed Jun 29 11:07:39 2022] POST /identity/v3/services => generated 242 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 11:07:39.448846 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e42467-4cdf-4fd3-8d7d-7e1db2731bf1 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:39.449139 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e42467-4cdf-4fd3-8d7d-7e1db2731bf1 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:39.449417 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e42467-4cdf-4fd3-8d7d-7e1db2731bf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:39.472646 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-58e42467-4cdf-4fd3-8d7d-7e1db2731bf1 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:07:39.473909 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-58e42467-4cdf-4fd3-8d7d-7e1db2731bf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:07:39.519258 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:39.536618 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:39.542523 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063b475e-6312-456b-b841-1a59f47fd1ef None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:39.550106 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 508/1011] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:07:39 2022] POST /identity/v3/auth/tokens => generated 1656 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 11:07:39.574426 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e42467-4cdf-4fd3-8d7d-7e1db2731bf1 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:39.588063 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c9e3da-6b06-44c1-86fc-f6ce22f96929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:39.588063 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c9e3da-6b06-44c1-86fc-f6ce22f96929 None admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:39.588063 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c9e3da-6b06-44c1-86fc-f6ce22f96929 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:39.588063 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c9e3da-6b06-44c1-86fc-f6ce22f96929 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:39.588532 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c9e3da-6b06-44c1-86fc-f6ce22f96929 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:39.588532 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c9e3da-6b06-44c1-86fc-f6ce22f96929 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:39.671910 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 504/1012] 10.222.0.44 () {60 vars in 1144 bytes} [Wed Jun 29 11:07:39 2022] POST /identity/v3/endpoints => generated 325 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:07:39.683489 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e42467-4cdf-4fd3-8d7d-7e1db2731bf1 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:39.752114 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a526f0a-3c92-44fb-bdc6-e88a7adbff9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:39.753662 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a526f0a-3c92-44fb-bdc6-e88a7adbff9f None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:39.754075 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a526f0a-3c92-44fb-bdc6-e88a7adbff9f None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:39.754482 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a526f0a-3c92-44fb-bdc6-e88a7adbff9f None admin] PATH_INFO: `/v3/services/compute` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:39.755290 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a526f0a-3c92-44fb-bdc6-e88a7adbff9f None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:39.756928 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a526f0a-3c92-44fb-bdc6-e88a7adbff9f None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:39.770419 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-9a526f0a-3c92-44fb-bdc6-e88a7adbff9f None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 29 11:07:39.770419 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:07:39.770419 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 29 11:07:39.770419 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 29 11:07:39.770419 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 29 11:07:39.770419 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 29 11:07:39.770419 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 29 11:07:39.770419 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 11:07:39.770419 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 29 11:07:39.770419 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jun 29 11:07:39.770419 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 29 11:07:39.770419 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jun 29 11:07:39.770419 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:07:39.770419 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:07:39.770419 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:07:39.770419 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:07:39.770419 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:07:39.771507 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:07:39.771507 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:07:39.771507 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:07:39.771507 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:07:39.771507 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:07:39.771507 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:07:39.771507 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 29 11:07:39.771507 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 29 11:07:39.771507 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 29 11:07:39.771507 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 29 11:07:39.771507 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:07:39.771507 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:07:39.771507 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:07:39.771507 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:07:39.771507 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:07:39.771507 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:07:39.773139 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:07:39.773139 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jun 29 11:07:39.773139 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:07:39.773139 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:07:39.773139 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:07:39.773139 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:07:39.773139 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:07:39.773139 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:07:39.773139 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:07:39.773139 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:07:39.773139 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 29 11:07:39.773139 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 11:07:39.773139 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 29 11:07:39.773139 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jun 29 11:07:39.773139 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 509/1013] 10.222.0.44 () {56 vars in 1111 bytes} [Wed Jun 29 11:07:39 2022] GET /identity/v3/services/compute => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:07:39.777774 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c9e3da-6b06-44c1-86fc-f6ce22f96929 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:39.889281 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c9e3da-6b06-44c1-86fc-f6ce22f96929 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:39.931143 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb50090-daf6-4e7c-bee7-71c64de57274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:39.931817 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb50090-daf6-4e7c-bee7-71c64de57274 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:39.932129 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb50090-daf6-4e7c-bee7-71c64de57274 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:39.932489 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb50090-daf6-4e7c-bee7-71c64de57274 None admin] PATH_INFO: `/v3/services` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:39.933197 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb50090-daf6-4e7c-bee7-71c64de57274 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:07:39.933507 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb50090-daf6-4e7c-bee7-71c64de57274 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:39.934920 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb50090-daf6-4e7c-bee7-71c64de57274 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:39.940758 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 505/1014] 10.222.0.44 () {56 vars in 1112 bytes} [Wed Jun 29 11:07:39 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:07:39.946444 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a526f0a-3c92-44fb-bdc6-e88a7adbff9f None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:40.000594 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:40.070376 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a526f0a-3c92-44fb-bdc6-e88a7adbff9f None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:40.103580 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abdc889-5269-4a40-8f30-206b5f4bcee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:40.104268 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abdc889-5269-4a40-8f30-206b5f4bcee0 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:40.104549 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abdc889-5269-4a40-8f30-206b5f4bcee0 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:40.104822 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abdc889-5269-4a40-8f30-206b5f4bcee0 None admin] PATH_INFO: `/v3/services` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:40.105526 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abdc889-5269-4a40-8f30-206b5f4bcee0 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:07:40.105832 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abdc889-5269-4a40-8f30-206b5f4bcee0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:40.107186 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abdc889-5269-4a40-8f30-206b5f4bcee0 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:40.117163 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 510/1015] 10.222.0.44 () {56 vars in 1112 bytes} [Wed Jun 29 11:07:39 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 11:07:40.118340 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb50090-daf6-4e7c-bee7-71c64de57274 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:40.128237 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:40.220527 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb50090-daf6-4e7c-bee7-71c64de57274 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:40.254432 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9119f48a-fada-47a4-b221-334d8d6cfaec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:40.255093 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9119f48a-fada-47a4-b221-334d8d6cfaec None admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:40.255375 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9119f48a-fada-47a4-b221-334d8d6cfaec None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:40.255650 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9119f48a-fada-47a4-b221-334d8d6cfaec None admin] PATH_INFO: `/v3/endpoints` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:40.256407 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9119f48a-fada-47a4-b221-334d8d6cfaec None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:40.257751 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9119f48a-fada-47a4-b221-334d8d6cfaec None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:40.368512 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 506/1016] 10.222.0.44 () {60 vars in 1144 bytes} [Wed Jun 29 11:07:40 2022] POST /identity/v3/endpoints => generated 334 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:07:41.533505 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164df23e-a502-4f8a-8598-06f8117fbef1 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:41.534016 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164df23e-a502-4f8a-8598-06f8117fbef1 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:41.534016 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164df23e-a502-4f8a-8598-06f8117fbef1 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:41.535539 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 511/1017] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:07:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:07:41.540432 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f39f4ba-bfda-49aa-a2cd-51c4828d65ac None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:41.540816 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f39f4ba-bfda-49aa-a2cd-51c4828d65ac None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:41.540992 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f39f4ba-bfda-49aa-a2cd-51c4828d65ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:41.558138 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-3f39f4ba-bfda-49aa-a2cd-51c4828d65ac None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:07:41.559322 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-3f39f4ba-bfda-49aa-a2cd-51c4828d65ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:07:41.623553 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:41.640856 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 507/1018] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:07:41 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 11:07:41.663374 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164df23e-a502-4f8a-8598-06f8117fbef1 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:41.766269 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164df23e-a502-4f8a-8598-06f8117fbef1 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:41.768070 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b0f8aa-db20-4b58-9b32-f4d9d2d25e14 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:41.768400 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b0f8aa-db20-4b58-9b32-f4d9d2d25e14 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:41.768684 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b0f8aa-db20-4b58-9b32-f4d9d2d25e14 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:41.770206 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 508/1019] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:07:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:07:41.774757 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c4d6d2-adc8-41d4-adbc-af8bcaf3dc7c None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:41.775057 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c4d6d2-adc8-41d4-adbc-af8bcaf3dc7c None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:41.775339 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c4d6d2-adc8-41d4-adbc-af8bcaf3dc7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:41.782644 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:41.794574 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-05c4d6d2-adc8-41d4-adbc-af8bcaf3dc7c None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:07:41.794574 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-05c4d6d2-adc8-41d4-adbc-af8bcaf3dc7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:07:41.805570 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5cf2a4-e1e3-4434-817b-0ec04577b232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:41.806487 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5cf2a4-e1e3-4434-817b-0ec04577b232 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:41.806788 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5cf2a4-e1e3-4434-817b-0ec04577b232 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:41.807062 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5cf2a4-e1e3-4434-817b-0ec04577b232 None admin] PATH_INFO: `/v3/services/image` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:41.808181 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5cf2a4-e1e3-4434-817b-0ec04577b232 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:41.809956 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5cf2a4-e1e3-4434-817b-0ec04577b232 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:41.826269 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-9c5cf2a4-e1e3-4434-817b-0ec04577b232 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 29 11:07:41.826269 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:07:41.826269 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 29 11:07:41.826269 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 29 11:07:41.826269 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 29 11:07:41.826269 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 29 11:07:41.826269 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 29 11:07:41.826269 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 11:07:41.826269 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 29 11:07:41.826269 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jun 29 11:07:41.826269 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 29 11:07:41.826269 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jun 29 11:07:41.826269 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:07:41.826269 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:07:41.826269 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:07:41.826269 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:07:41.826269 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:07:41.827252 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:07:41.827252 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:07:41.827252 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:07:41.827252 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:07:41.827252 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:07:41.827252 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:07:41.827252 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 29 11:07:41.827252 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 29 11:07:41.827252 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 29 11:07:41.827252 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 29 11:07:41.827252 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:07:41.827252 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:07:41.827252 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:07:41.827252 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:07:41.827252 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:07:41.827252 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:07:41.833262 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:07:41.833262 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jun 29 11:07:41.833262 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:07:41.833262 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:07:41.833262 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:07:41.833262 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:07:41.833262 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:07:41.833262 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:07:41.833262 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:07:41.833262 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:07:41.833262 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 29 11:07:41.833262 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 11:07:41.833262 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 29 11:07:41.833262 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jun 29 11:07:41.833262 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 512/1020] 10.222.0.44 () {56 vars in 1105 bytes} [Wed Jun 29 11:07:41 2022] GET /identity/v3/services/image => generated 86 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:07:41.834073 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5cf2a4-e1e3-4434-817b-0ec04577b232 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:41.862153 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 509/1021] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:07:41 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 29 11:07:41.885703 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c4d6d2-adc8-41d4-adbc-af8bcaf3dc7c None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:41.938941 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5cf2a4-e1e3-4434-817b-0ec04577b232 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:41.973645 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82a90f6-a840-465b-b80e-177d743e31ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:41.974300 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82a90f6-a840-465b-b80e-177d743e31ae None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:41.974581 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82a90f6-a840-465b-b80e-177d743e31ae None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:41.974851 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82a90f6-a840-465b-b80e-177d743e31ae None admin] PATH_INFO: `/v3/services` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:41.975545 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82a90f6-a840-465b-b80e-177d743e31ae None admin] RBAC: Adding query filter params (name=image) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:07:41.975869 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82a90f6-a840-465b-b80e-177d743e31ae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:41.977218 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82a90f6-a840-465b-b80e-177d743e31ae None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:41.986682 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 513/1022] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jun 29 11:07:41 2022] GET /identity/v3/services?name=image => generated 122 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 11:07:41.987725 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82a90f6-a840-465b-b80e-177d743e31ae None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:42.000443 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c4d6d2-adc8-41d4-adbc-af8bcaf3dc7c None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:42.020642 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:42.031638 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:42.040168 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7489c542-9f7e-4180-85f1-b484c43b9e2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:42.041236 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7489c542-9f7e-4180-85f1-b484c43b9e2f None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:42.041236 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7489c542-9f7e-4180-85f1-b484c43b9e2f None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:42.041513 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7489c542-9f7e-4180-85f1-b484c43b9e2f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:42.042692 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7489c542-9f7e-4180-85f1-b484c43b9e2f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:42.044301 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7489c542-9f7e-4180-85f1-b484c43b9e2f None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:42.046040 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 510/1023] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jun 29 11:07:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 11:07:42.052135 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7489c542-9f7e-4180-85f1-b484c43b9e2f None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:42.090495 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82a90f6-a840-465b-b80e-177d743e31ae None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:42.128963 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a853aa0-5498-4842-a9ed-b498ff195bf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:42.129635 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a853aa0-5498-4842-a9ed-b498ff195bf8 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:42.129920 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a853aa0-5498-4842-a9ed-b498ff195bf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:42.130216 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a853aa0-5498-4842-a9ed-b498ff195bf8 None admin] PATH_INFO: `/v3/services` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:42.130899 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a853aa0-5498-4842-a9ed-b498ff195bf8 None admin] RBAC: Adding query filter params (type=image) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:07:42.131528 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a853aa0-5498-4842-a9ed-b498ff195bf8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:42.132723 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a853aa0-5498-4842-a9ed-b498ff195bf8 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:42.142309 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 514/1024] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jun 29 11:07:41 2022] GET /identity/v3/services?type=image => generated 350 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 11:07:42.143486 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a853aa0-5498-4842-a9ed-b498ff195bf8 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:42.176150 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7489c542-9f7e-4180-85f1-b484c43b9e2f None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:42.214393 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc34de1-2d74-4481-b577-569874cca38e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:42.215252 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc34de1-2d74-4481-b577-569874cca38e None admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:42.215653 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc34de1-2d74-4481-b577-569874cca38e None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:42.215965 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc34de1-2d74-4481-b577-569874cca38e None admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:42.216672 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc34de1-2d74-4481-b577-569874cca38e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:42.218096 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc34de1-2d74-4481-b577-569874cca38e None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:42.257022 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a853aa0-5498-4842-a9ed-b498ff195bf8 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:42.263738 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:42.289292 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2adea5-871b-49e7-878e-0d5c278ae489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:42.289945 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2adea5-871b-49e7-878e-0d5c278ae489 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:42.290223 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2adea5-871b-49e7-878e-0d5c278ae489 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:42.290499 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2adea5-871b-49e7-878e-0d5c278ae489 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:42.291321 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2adea5-871b-49e7-878e-0d5c278ae489 None admin] RBAC: Adding query filter params (interface=public, service_id=4eb76ef2af1d4b09b64880f5cb58bb7d, region_id=RegionOne) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:07:42.291627 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2adea5-871b-49e7-878e-0d5c278ae489 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:42.293031 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2adea5-871b-49e7-878e-0d5c278ae489 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:42.296572 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 511/1025] 10.222.0.44 () {60 vars in 1141 bytes} [Wed Jun 29 11:07:42 2022] POST /identity/v3/projects => generated 311 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 11:07:42.303243 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 515/1026] 10.222.0.44 () {56 vars in 1251 bytes} [Wed Jun 29 11:07:42 2022] GET /identity/v3/endpoints?service_id=4eb76ef2af1d4b09b64880f5cb58bb7d&interface=public®ion_id=RegionOne => generated 194 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 11:07:42.304777 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc34de1-2d74-4481-b577-569874cca38e None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:42.352035 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:42.417797 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc34de1-2d74-4481-b577-569874cca38e None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:42.454922 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690b3497-b6dc-49a2-b988-25f8897d55d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:42.456144 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690b3497-b6dc-49a2-b988-25f8897d55d7 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:42.456546 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690b3497-b6dc-49a2-b988-25f8897d55d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:42.457982 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690b3497-b6dc-49a2-b988-25f8897d55d7 None admin] PATH_INFO: `/v3/services` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:42.457982 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690b3497-b6dc-49a2-b988-25f8897d55d7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:42.460323 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690b3497-b6dc-49a2-b988-25f8897d55d7 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:42.470086 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 512/1027] 10.222.0.44 () {56 vars in 1087 bytes} [Wed Jun 29 11:07:42 2022] GET /identity/v3/services => generated 1957 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 11:07:42.506446 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bc0afe-97a3-4d80-a978-03b32694a90e None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:42.506741 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bc0afe-97a3-4d80-a978-03b32694a90e None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:42.507005 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bc0afe-97a3-4d80-a978-03b32694a90e None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:42.508631 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 516/1028] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:07:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:07:42.514532 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afcd07c-526f-49ee-ba6b-8758ed52c1c5 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:42.514907 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afcd07c-526f-49ee-ba6b-8758ed52c1c5 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:42.515269 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afcd07c-526f-49ee-ba6b-8758ed52c1c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:42.535987 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-2afcd07c-526f-49ee-ba6b-8758ed52c1c5 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:07:42.537234 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-2afcd07c-526f-49ee-ba6b-8758ed52c1c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:07:42.612489 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 513/1029] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:07:42 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 11:07:42.636738 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bc0afe-97a3-4d80-a978-03b32694a90e None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:42.740349 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bc0afe-97a3-4d80-a978-03b32694a90e None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:42.773907 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193e66b5-af39-4eee-a5d5-77456439b7f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:42.774724 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193e66b5-af39-4eee-a5d5-77456439b7f6 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:42.775006 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193e66b5-af39-4eee-a5d5-77456439b7f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:42.775288 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193e66b5-af39-4eee-a5d5-77456439b7f6 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:42.786441 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193e66b5-af39-4eee-a5d5-77456439b7f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 11:07:42.794626 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-193e66b5-af39-4eee-a5d5-77456439b7f6 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 29 11:07:42.794626 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:07:42.794626 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:07:42.794626 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:07:42.794626 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:07:42.794626 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:07:42.794626 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:07:42.794626 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:07:42.794626 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:07:42.794626 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:07:42.794626 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:07:42.794626 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:07:42.794626 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 11:07:42.794626 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 11:07:42.794626 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 11:07:42.794626 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 11:07:42.795549 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 11:07:42.795549 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 11:07:42.795549 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 11:07:42.795549 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 11:07:42.795549 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:07:42.795549 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:07:42.795549 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:07:42.795549 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:07:42.795549 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:07:42.795549 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:07:42.795549 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:07:42.795549 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jun 29 11:07:42.795549 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:07:42.795549 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:07:42.795549 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:07:42.795549 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:07:42.796880 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:07:42.796880 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:07:42.796880 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:07:42.796880 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:07:42.796880 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 11:07:42.796880 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 11:07:42.796880 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 11:07:42.796880 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 11:07:42.796880 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 11:07:42.796880 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 11:07:42.796880 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 29 11:07:42.796880 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jun 29 11:07:42.800149 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 517/1030] 10.222.0.44 () {56 vars in 1120 bytes} [Wed Jun 29 11:07:42 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:07:42.801760 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afcd07c-526f-49ee-ba6b-8758ed52c1c5 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:42.836137 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:42.905579 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afcd07c-526f-49ee-ba6b-8758ed52c1c5 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:42.946461 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edba8dd-53d1-47f6-a1f3-2d88e5c34ee7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:42.947142 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edba8dd-53d1-47f6-a1f3-2d88e5c34ee7 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:42.947424 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edba8dd-53d1-47f6-a1f3-2d88e5c34ee7 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:42.947698 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edba8dd-53d1-47f6-a1f3-2d88e5c34ee7 None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:42.948481 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edba8dd-53d1-47f6-a1f3-2d88e5c34ee7 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:07:42.948845 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edba8dd-53d1-47f6-a1f3-2d88e5c34ee7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:42.950197 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edba8dd-53d1-47f6-a1f3-2d88e5c34ee7 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:42.959874 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 514/1031] 10.222.0.44 () {56 vars in 1115 bytes} [Wed Jun 29 11:07:42 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:07:42.964459 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193e66b5-af39-4eee-a5d5-77456439b7f6 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:42.995980 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:43.066051 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193e66b5-af39-4eee-a5d5-77456439b7f6 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:43.099315 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bbb55f-1ae0-4796-af4c-beb7b5d2cc2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:43.100140 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bbb55f-1ae0-4796-af4c-beb7b5d2cc2b None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:43.100427 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bbb55f-1ae0-4796-af4c-beb7b5d2cc2b None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:43.100702 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bbb55f-1ae0-4796-af4c-beb7b5d2cc2b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:43.110863 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bbb55f-1ae0-4796-af4c-beb7b5d2cc2b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:43.112904 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bbb55f-1ae0-4796-af4c-beb7b5d2cc2b None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:43.130976 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-d2bbb55f-1ae0-4796-af4c-beb7b5d2cc2b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 11:07:43.130976 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:07:43.130976 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:07:43.130976 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:07:43.130976 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:07:43.130976 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:07:43.130976 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:07:43.130976 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:07:43.130976 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:07:43.130976 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:07:43.130976 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:07:43.130976 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:07:43.130976 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 11:07:43.130976 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 11:07:43.130976 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 11:07:43.132193 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 11:07:43.132193 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:07:43.132193 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:07:43.132193 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:07:43.132193 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:07:43.132193 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:07:43.132193 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:07:43.132193 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:07:43.132193 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jun 29 11:07:43.132193 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:07:43.132193 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:07:43.132193 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:07:43.132193 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:07:43.132193 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:07:43.132193 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:07:43.132193 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:07:43.132193 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:07:43.137673 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 11:07:43.137673 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 11:07:43.137673 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 11:07:43.137673 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jun 29 11:07:43.137673 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 518/1032] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jun 29 11:07:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 11:07:43.137968 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edba8dd-53d1-47f6-a1f3-2d88e5c34ee7 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:43.235705 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:43.243339 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edba8dd-53d1-47f6-a1f3-2d88e5c34ee7 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:43.278856 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d443d543-8971-47f4-ae0b-5f27865a3a6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:43.279522 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d443d543-8971-47f4-ae0b-5f27865a3a6d None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:43.279808 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d443d543-8971-47f4-ae0b-5f27865a3a6d None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:43.280115 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d443d543-8971-47f4-ae0b-5f27865a3a6d None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:43.280823 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d443d543-8971-47f4-ae0b-5f27865a3a6d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:07:43.281130 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d443d543-8971-47f4-ae0b-5f27865a3a6d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:43.282521 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d443d543-8971-47f4-ae0b-5f27865a3a6d None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:43.293787 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 515/1033] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Jun 29 11:07:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 11:07:43.299134 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bbb55f-1ae0-4796-af4c-beb7b5d2cc2b None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:43.381239 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:43.402210 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bbb55f-1ae0-4796-af4c-beb7b5d2cc2b None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:43.437491 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbcd782-f017-4043-924c-630c1b14ffe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:43.438281 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbcd782-f017-4043-924c-630c1b14ffe9 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:43.438569 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbcd782-f017-4043-924c-630c1b14ffe9 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:43.438849 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbcd782-f017-4043-924c-630c1b14ffe9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:43.448308 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbcd782-f017-4043-924c-630c1b14ffe9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:43.449798 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbcd782-f017-4043-924c-630c1b14ffe9 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:43.465350 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-0bbcd782-f017-4043-924c-630c1b14ffe9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 29 11:07:43.465350 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:07:43.465350 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:07:43.465350 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:07:43.465350 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:07:43.465350 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:07:43.465350 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:07:43.465350 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:07:43.465350 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:07:43.465350 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:07:43.465350 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:07:43.465350 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:07:43.465350 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 11:07:43.465350 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 11:07:43.465350 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 11:07:43.465350 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 11:07:43.466381 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:07:43.466381 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:07:43.466381 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 11:07:43.466381 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 11:07:43.466381 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 11:07:43.466381 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 11:07:43.466381 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:07:43.466381 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:07:43.466381 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:07:43.466381 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:07:43.466381 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:07:43.466381 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jun 29 11:07:43.466381 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:07:43.466381 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:07:43.466381 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:07:43.466381 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:07:43.466381 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:07:43.466381 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:07:43.467347 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:07:43.467347 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:07:43.467347 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 11:07:43.467347 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 11:07:43.467347 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 11:07:43.467347 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 11:07:43.467347 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 11:07:43.467347 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 11:07:43.467347 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 29 11:07:43.467347 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jun 29 11:07:43.467347 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 519/1034] 10.222.0.44 () {56 vars in 1093 bytes} [Wed Jun 29 11:07:43 2022] GET /identity/v3/users/nova => generated 82 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:07:43.472847 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d443d543-8971-47f4-ae0b-5f27865a3a6d None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:43.575288 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d443d543-8971-47f4-ae0b-5f27865a3a6d None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:43.610516 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f530f1-e3d4-475c-bf34-4bc508f3ea12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:43.611188 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f530f1-e3d4-475c-bf34-4bc508f3ea12 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:43.611472 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f530f1-e3d4-475c-bf34-4bc508f3ea12 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:43.611743 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f530f1-e3d4-475c-bf34-4bc508f3ea12 None admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:43.612622 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f530f1-e3d4-475c-bf34-4bc508f3ea12 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:07:43.613163 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f530f1-e3d4-475c-bf34-4bc508f3ea12 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:43.614333 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f530f1-e3d4-475c-bf34-4bc508f3ea12 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:43.632263 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 516/1035] 10.222.0.44 () {56 vars in 1133 bytes} [Wed Jun 29 11:07:43 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:07:43.637685 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbcd782-f017-4043-924c-630c1b14ffe9 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:43.736273 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbcd782-f017-4043-924c-630c1b14ffe9 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:43.769618 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5fa4ff-58b7-4c90-9ad8-581b2ca0d601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:43.770262 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5fa4ff-58b7-4c90-9ad8-581b2ca0d601 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:43.770550 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5fa4ff-58b7-4c90-9ad8-581b2ca0d601 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:43.770825 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5fa4ff-58b7-4c90-9ad8-581b2ca0d601 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:43.782432 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:43.838391 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5fa4ff-58b7-4c90-9ad8-581b2ca0d601 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:43.840091 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5fa4ff-58b7-4c90-9ad8-581b2ca0d601 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:43.943669 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 520/1036] 10.222.0.44 () {58 vars in 1282 bytes} [Wed Jun 29 11:07:43 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 29 11:07:43.949247 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f530f1-e3d4-475c-bf34-4bc508f3ea12 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:43.964238 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:44.050463 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f530f1-e3d4-475c-bf34-4bc508f3ea12 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:44.085020 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3077afe8-12c6-4076-898c-ac57f161bf83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:44.085813 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3077afe8-12c6-4076-898c-ac57f161bf83 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:44.086095 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3077afe8-12c6-4076-898c-ac57f161bf83 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:44.086371 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3077afe8-12c6-4076-898c-ac57f161bf83 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:44.097206 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3077afe8-12c6-4076-898c-ac57f161bf83 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:44.098644 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3077afe8-12c6-4076-898c-ac57f161bf83 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:44.116583 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-3077afe8-12c6-4076-898c-ac57f161bf83 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 11:07:44.116583 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:07:44.116583 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:07:44.116583 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:07:44.116583 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:07:44.116583 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:07:44.116583 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:07:44.116583 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:07:44.116583 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:07:44.116583 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:07:44.116583 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:07:44.116583 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:07:44.116583 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 11:07:44.116583 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 11:07:44.116583 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 11:07:44.117507 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 11:07:44.117507 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:07:44.117507 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:07:44.117507 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:07:44.117507 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:07:44.117507 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:07:44.117507 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:07:44.117507 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:07:44.117507 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jun 29 11:07:44.117507 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:07:44.117507 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:07:44.117507 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:07:44.117507 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:07:44.117507 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:07:44.117507 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:07:44.117507 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:07:44.117507 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:07:44.123080 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 11:07:44.123080 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 11:07:44.123080 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 11:07:44.123080 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:07:44.123080 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 517/1037] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jun 29 11:07:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:07:44.123683 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5fa4ff-58b7-4c90-9ad8-581b2ca0d601 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:44.168769 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:44.226884 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5fa4ff-58b7-4c90-9ad8-581b2ca0d601 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:44.260742 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de221d19-5514-49f0-b610-78f6bdc88d9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:44.261407 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de221d19-5514-49f0-b610-78f6bdc88d9e None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:44.261693 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de221d19-5514-49f0-b610-78f6bdc88d9e None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:44.261970 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de221d19-5514-49f0-b610-78f6bdc88d9e None admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:44.262691 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de221d19-5514-49f0-b610-78f6bdc88d9e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:07:44.263001 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de221d19-5514-49f0-b610-78f6bdc88d9e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:44.264407 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de221d19-5514-49f0-b610-78f6bdc88d9e None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:44.276522 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 521/1038] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Jun 29 11:07:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:07:44.281867 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3077afe8-12c6-4076-898c-ac57f161bf83 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:44.344718 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:44.384499 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3077afe8-12c6-4076-898c-ac57f161bf83 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:44.392286 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1e6c73-9961-4cca-b4be-a10315e2063e None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:44.392580 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1e6c73-9961-4cca-b4be-a10315e2063e None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:44.392863 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1e6c73-9961-4cca-b4be-a10315e2063e None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:44.394802 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 522/1039] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:07:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 29 11:07:44.399878 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ac9d0b-5d4d-4780-a879-fa949009b320 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:44.400176 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ac9d0b-5d4d-4780-a879-fa949009b320 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:44.400462 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ac9d0b-5d4d-4780-a879-fa949009b320 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:44.416983 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-80ac9d0b-5d4d-4780-a879-fa949009b320 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:07:44.418223 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-80ac9d0b-5d4d-4780-a879-fa949009b320 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:07:44.420670 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec16ebb-a79e-41ee-b3be-36cc3fba6436 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:44.421583 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec16ebb-a79e-41ee-b3be-36cc3fba6436 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:44.421886 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec16ebb-a79e-41ee-b3be-36cc3fba6436 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:44.422147 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec16ebb-a79e-41ee-b3be-36cc3fba6436 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:44.429349 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec16ebb-a79e-41ee-b3be-36cc3fba6436 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:44.430776 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec16ebb-a79e-41ee-b3be-36cc3fba6436 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:44.445306 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-aec16ebb-a79e-41ee-b3be-36cc3fba6436 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 11:07:44.445306 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:07:44.445306 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:07:44.445306 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:07:44.445306 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:07:44.445306 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:07:44.445306 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:07:44.445306 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:07:44.445306 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:07:44.445306 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:07:44.445306 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:07:44.445306 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:07:44.445306 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 11:07:44.445306 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 11:07:44.445306 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 11:07:44.446464 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 11:07:44.446464 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:07:44.446464 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:07:44.446464 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:07:44.446464 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:07:44.446464 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:07:44.446464 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:07:44.446464 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:07:44.446464 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jun 29 11:07:44.446464 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:07:44.446464 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:07:44.446464 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:07:44.446464 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:07:44.446464 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:07:44.446464 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:07:44.446464 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:07:44.446464 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:07:44.447390 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 11:07:44.447390 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 11:07:44.447390 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 11:07:44.447390 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 11:07:44.447390 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 11:07:44.447390 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 11:07:44.447390 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 11:07:44.447390 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:07:44.447390 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 518/1040] 10.222.0.44 () {56 vars in 1111 bytes} [Wed Jun 29 11:07:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:07:44.452351 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec16ebb-a79e-41ee-b3be-36cc3fba6436 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:44.497793 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 523/1041] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:07:44 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 29 11:07:44.503598 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1097d23-4287-4149-843c-d22d3113322f None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:44.503911 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1097d23-4287-4149-843c-d22d3113322f None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:44.504196 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1097d23-4287-4149-843c-d22d3113322f None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:44.505580 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 524/1042] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:07:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 29 11:07:44.510218 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d2f067-bb2c-4a37-8920-7a76e0cb3ac7 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:44.510508 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d2f067-bb2c-4a37-8920-7a76e0cb3ac7 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:44.510787 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d2f067-bb2c-4a37-8920-7a76e0cb3ac7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:44.527158 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-39d2f067-bb2c-4a37-8920-7a76e0cb3ac7 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:07:44.528370 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-39d2f067-bb2c-4a37-8920-7a76e0cb3ac7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:07:44.553747 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:44.572989 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec16ebb-a79e-41ee-b3be-36cc3fba6436 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:44.593982 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 525/1043] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:07:44 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 29 11:07:44.594798 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d2f067-bb2c-4a37-8920-7a76e0cb3ac7 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:44.595602 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:44.607355 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32a597b-8e90-42e7-80d6-0cfa367324cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:44.608045 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32a597b-8e90-42e7-80d6-0cfa367324cd None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:44.608339 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32a597b-8e90-42e7-80d6-0cfa367324cd None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:44.608651 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32a597b-8e90-42e7-80d6-0cfa367324cd None admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:44.609452 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32a597b-8e90-42e7-80d6-0cfa367324cd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:07:44.609766 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32a597b-8e90-42e7-80d6-0cfa367324cd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:44.611196 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32a597b-8e90-42e7-80d6-0cfa367324cd None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:44.624616 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 519/1044] 10.222.0.44 () {56 vars in 1148 bytes} [Wed Jun 29 11:07:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 11:07:44.625424 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32a597b-8e90-42e7-80d6-0cfa367324cd None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:44.693966 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d2f067-bb2c-4a37-8920-7a76e0cb3ac7 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:44.726278 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32a597b-8e90-42e7-80d6-0cfa367324cd None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:44.726930 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b3c663-17ae-42f1-8b96-964488ffdb1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:44.727725 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b3c663-17ae-42f1-8b96-964488ffdb1b None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:44.728040 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b3c663-17ae-42f1-8b96-964488ffdb1b None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:44.728316 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b3c663-17ae-42f1-8b96-964488ffdb1b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:44.729487 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b3c663-17ae-42f1-8b96-964488ffdb1b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:44.730916 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b3c663-17ae-42f1-8b96-964488ffdb1b None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:44.732544 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 526/1045] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jun 29 11:07:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:07:44.733299 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b3c663-17ae-42f1-8b96-964488ffdb1b None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:44.760889 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93f5019-3f33-4ae5-930a-837acf81dce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:44.761668 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93f5019-3f33-4ae5-930a-837acf81dce1 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:44.761950 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93f5019-3f33-4ae5-930a-837acf81dce1 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:44.762222 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93f5019-3f33-4ae5-930a-837acf81dce1 None admin] PATH_INFO: `/v3/services/image` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:44.762867 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93f5019-3f33-4ae5-930a-837acf81dce1 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:44.763170 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:44.764271 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93f5019-3f33-4ae5-930a-837acf81dce1 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:44.775532 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-b93f5019-3f33-4ae5-930a-837acf81dce1 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 29 11:07:44.775532 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:07:44.775532 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 29 11:07:44.775532 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 29 11:07:44.775532 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 29 11:07:44.775532 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 29 11:07:44.775532 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 29 11:07:44.775532 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 11:07:44.775532 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 29 11:07:44.775532 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:07:44.775532 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 29 11:07:44.775532 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:07:44.775532 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:07:44.775532 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:07:44.775532 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:07:44.775532 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:07:44.775532 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:07:44.776589 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:07:44.776589 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:07:44.776589 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:07:44.776589 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:07:44.776589 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:07:44.776589 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:07:44.776589 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 29 11:07:44.776589 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 29 11:07:44.776589 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 29 11:07:44.776589 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 29 11:07:44.776589 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:07:44.776589 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:07:44.776589 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:07:44.776589 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:07:44.776589 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:07:44.776589 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:07:44.777464 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:07:44.777464 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jun 29 11:07:44.777464 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:07:44.777464 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:07:44.777464 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:07:44.777464 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:07:44.777464 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:07:44.777464 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:07:44.777464 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:07:44.777464 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:07:44.777464 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 29 11:07:44.777464 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 11:07:44.777464 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 29 11:07:44.777464 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:07:44.777464 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 520/1046] 10.222.0.44 () {56 vars in 1105 bytes} [Wed Jun 29 11:07:44 2022] GET /identity/v3/services/image => generated 86 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:07:44.778294 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93f5019-3f33-4ae5-930a-837acf81dce1 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:44.826291 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:44.833391 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b3c663-17ae-42f1-8b96-964488ffdb1b None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:44.867508 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c5e195-c38f-447c-b384-33d1b3456fe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:44.868192 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c5e195-c38f-447c-b384-33d1b3456fe3 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:44.868477 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c5e195-c38f-447c-b384-33d1b3456fe3 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:44.868752 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c5e195-c38f-447c-b384-33d1b3456fe3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:44.869575 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c5e195-c38f-447c-b384-33d1b3456fe3 None admin] RBAC: Adding query filter params (role.id=3e8ef91b147f4b6b803ef07f3301be3f, scope.project.id=ebe5dffa4bb8440286687daefb6c98f4, user.id=796aeeff16df4739bddec1f6dee61bfe) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:07:44.869887 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c5e195-c38f-447c-b384-33d1b3456fe3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:44.871314 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c5e195-c38f-447c-b384-33d1b3456fe3 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:44.876425 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93f5019-3f33-4ae5-930a-837acf81dce1 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:44.876693 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 527/1047] 10.222.0.44 () {56 vars in 1374 bytes} [Wed Jun 29 11:07:44 2022] GET /identity/v3/role_assignments?user.id=796aeeff16df4739bddec1f6dee61bfe&scope.project.id=ebe5dffa4bb8440286687daefb6c98f4&role.id=3e8ef91b147f4b6b803ef07f3301be3f => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:07:44.877464 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c5e195-c38f-447c-b384-33d1b3456fe3 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:44.930625 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4d062c-80d0-47db-9236-150b6070acd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:44.931332 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4d062c-80d0-47db-9236-150b6070acd4 None admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:44.931618 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4d062c-80d0-47db-9236-150b6070acd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:44.931933 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4d062c-80d0-47db-9236-150b6070acd4 None admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:44.932680 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4d062c-80d0-47db-9236-150b6070acd4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:44.934157 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4d062c-80d0-47db-9236-150b6070acd4 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:44.986828 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:44.994185 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c5e195-c38f-447c-b384-33d1b3456fe3 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:45.028299 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7bc8b0-22fc-4392-b09b-059d7c2a5aef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:45.028966 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7bc8b0-22fc-4392-b09b-059d7c2a5aef None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:45.029248 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7bc8b0-22fc-4392-b09b-059d7c2a5aef None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:45.029523 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7bc8b0-22fc-4392-b09b-059d7c2a5aef None admin] PATH_INFO: `/v3/services` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:45.030230 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7bc8b0-22fc-4392-b09b-059d7c2a5aef None admin] RBAC: Adding query filter params (name=image) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:07:45.030535 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7bc8b0-22fc-4392-b09b-059d7c2a5aef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:45.031936 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7bc8b0-22fc-4392-b09b-059d7c2a5aef None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:45.037261 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 528/1048] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jun 29 11:07:44 2022] GET /identity/v3/services?name=image => generated 122 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:07:45.043193 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7bc8b0-22fc-4392-b09b-059d7c2a5aef None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:45.055865 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 521/1049] 10.222.0.44 () {60 vars in 1132 bytes} [Wed Jun 29 11:07:44 2022] POST /identity/v3/users => generated 285 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:07:45.154306 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7bc8b0-22fc-4392-b09b-059d7c2a5aef None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:45.191572 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f598e9b3-55eb-4840-a016-e6e2bfe69949 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:45.192285 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f598e9b3-55eb-4840-a016-e6e2bfe69949 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:45.192572 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f598e9b3-55eb-4840-a016-e6e2bfe69949 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:45.192852 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f598e9b3-55eb-4840-a016-e6e2bfe69949 None admin] PATH_INFO: `/v3/services` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:45.193573 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f598e9b3-55eb-4840-a016-e6e2bfe69949 None admin] RBAC: Adding query filter params (type=image) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:07:45.193881 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f598e9b3-55eb-4840-a016-e6e2bfe69949 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:45.195258 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f598e9b3-55eb-4840-a016-e6e2bfe69949 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:45.200906 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 529/1050] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jun 29 11:07:45 2022] GET /identity/v3/services?type=image => generated 350 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 11:07:45.206733 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4d062c-80d0-47db-9236-150b6070acd4 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:45.318911 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4d062c-80d0-47db-9236-150b6070acd4 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:45.322609 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:45.355468 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbde45a2-ccdc-4414-8ba4-853f1bfec3f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:45.356287 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbde45a2-ccdc-4414-8ba4-853f1bfec3f4 None admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:45.356831 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbde45a2-ccdc-4414-8ba4-853f1bfec3f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:45.357577 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbde45a2-ccdc-4414-8ba4-853f1bfec3f4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:45.359234 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbde45a2-ccdc-4414-8ba4-853f1bfec3f4 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:45.360870 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbde45a2-ccdc-4414-8ba4-853f1bfec3f4 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:45.458087 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 522/1051] 10.222.0.44 () {60 vars in 1144 bytes} [Wed Jun 29 11:07:45 2022] POST /identity/v3/endpoints => generated 327 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:07:46.894827 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1098ad-09ec-452b-b66f-bb9499ed472f None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:46.895370 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1098ad-09ec-452b-b66f-bb9499ed472f None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:46.895370 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1098ad-09ec-452b-b66f-bb9499ed472f None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:46.896799 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 530/1052] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:07:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:07:46.901893 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41ec0f3-2e01-4b08-b16f-2d98608f6c5e None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:46.902265 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41ec0f3-2e01-4b08-b16f-2d98608f6c5e None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:46.902461 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41ec0f3-2e01-4b08-b16f-2d98608f6c5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:46.920710 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-f41ec0f3-2e01-4b08-b16f-2d98608f6c5e None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:07:46.921891 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-f41ec0f3-2e01-4b08-b16f-2d98608f6c5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:07:46.994554 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 523/1053] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:07:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 11:07:47.016257 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1098ad-09ec-452b-b66f-bb9499ed472f None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:47.042315 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:47.119138 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1098ad-09ec-452b-b66f-bb9499ed472f None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:47.122414 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb4ce15-8308-4b79-8786-3e0f9bd32503 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:47.122774 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb4ce15-8308-4b79-8786-3e0f9bd32503 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:47.123124 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb4ce15-8308-4b79-8786-3e0f9bd32503 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:47.124604 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 524/1054] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:07:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 29 11:07:47.130189 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a445c8f-dbf7-4bfd-802d-4b86ab27842b None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:47.130476 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a445c8f-dbf7-4bfd-802d-4b86ab27842b None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:47.130747 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a445c8f-dbf7-4bfd-802d-4b86ab27842b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:47.148375 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-9a445c8f-dbf7-4bfd-802d-4b86ab27842b None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:07:47.149500 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-9a445c8f-dbf7-4bfd-802d-4b86ab27842b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:07:47.160414 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cbfb14-7c86-41e0-9df0-18f35be20bc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:47.161228 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cbfb14-7c86-41e0-9df0-18f35be20bc6 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:47.161510 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cbfb14-7c86-41e0-9df0-18f35be20bc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:47.161784 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cbfb14-7c86-41e0-9df0-18f35be20bc6 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:47.173432 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cbfb14-7c86-41e0-9df0-18f35be20bc6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 11:07:47.181678 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-97cbfb14-7c86-41e0-9df0-18f35be20bc6 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 29 11:07:47.181678 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:07:47.181678 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:07:47.181678 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:07:47.181678 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:07:47.181678 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:07:47.181678 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:07:47.181678 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:07:47.181678 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:07:47.181678 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:07:47.181678 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:07:47.181678 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:07:47.181678 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 11:07:47.181678 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 11:07:47.181678 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 11:07:47.181678 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 11:07:47.182622 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 11:07:47.182622 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 11:07:47.182622 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 11:07:47.182622 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 11:07:47.182622 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:07:47.182622 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:07:47.182622 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:07:47.182622 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:07:47.182622 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:07:47.182622 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:07:47.182622 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:07:47.182622 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jun 29 11:07:47.182622 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:07:47.182622 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:07:47.182622 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:07:47.182622 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:07:47.184147 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:07:47.184147 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:07:47.184147 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:07:47.184147 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:07:47.184147 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 11:07:47.184147 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 11:07:47.184147 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 11:07:47.184147 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 11:07:47.184147 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 11:07:47.184147 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 11:07:47.184147 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 29 11:07:47.184147 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jun 29 11:07:47.184147 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 531/1055] 10.222.0.44 () {56 vars in 1120 bytes} [Wed Jun 29 11:07:47 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:07:47.188944 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cbfb14-7c86-41e0-9df0-18f35be20bc6 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:47.221949 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 525/1056] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:07:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 29 11:07:47.244951 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a445c8f-dbf7-4bfd-802d-4b86ab27842b None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:47.263412 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:47.278353 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:47.295517 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cbfb14-7c86-41e0-9df0-18f35be20bc6 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:47.332780 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9a4f6f-927e-46c6-860c-4cccb8943ea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:47.333843 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9a4f6f-927e-46c6-860c-4cccb8943ea1 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:47.334131 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9a4f6f-927e-46c6-860c-4cccb8943ea1 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:47.334407 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9a4f6f-927e-46c6-860c-4cccb8943ea1 None admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:47.335551 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9a4f6f-927e-46c6-860c-4cccb8943ea1 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:07:47.335886 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9a4f6f-927e-46c6-860c-4cccb8943ea1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:47.337881 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9a4f6f-927e-46c6-860c-4cccb8943ea1 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:47.349347 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 532/1057] 10.222.0.44 () {56 vars in 1115 bytes} [Wed Jun 29 11:07:47 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:07:47.354797 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9a4f6f-927e-46c6-860c-4cccb8943ea1 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:47.357108 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a445c8f-dbf7-4bfd-802d-4b86ab27842b None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:47.393599 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5933b82d-54cc-48c1-a800-e4c81d538b21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:47.394404 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5933b82d-54cc-48c1-a800-e4c81d538b21 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:47.395026 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5933b82d-54cc-48c1-a800-e4c81d538b21 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:47.395026 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5933b82d-54cc-48c1-a800-e4c81d538b21 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:47.405015 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5933b82d-54cc-48c1-a800-e4c81d538b21 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 11:07:47.412634 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-5933b82d-54cc-48c1-a800-e4c81d538b21 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 11:07:47.412634 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:07:47.412634 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:07:47.412634 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:07:47.412634 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:07:47.412634 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:07:47.412634 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:07:47.412634 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:07:47.412634 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:07:47.412634 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:07:47.412634 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:07:47.412634 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:07:47.412634 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 11:07:47.412634 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 11:07:47.412634 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 11:07:47.412634 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 11:07:47.413604 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 11:07:47.413604 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 11:07:47.413604 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 11:07:47.413604 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 11:07:47.413604 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:07:47.413604 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:07:47.413604 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:07:47.413604 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:07:47.413604 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:07:47.413604 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:07:47.413604 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:07:47.413604 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jun 29 11:07:47.413604 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:07:47.413604 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:07:47.413604 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:07:47.413604 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:07:47.414466 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:07:47.414466 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:07:47.414466 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:07:47.414466 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:07:47.414466 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 11:07:47.414466 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 11:07:47.414466 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 11:07:47.414466 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 11:07:47.414466 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 11:07:47.414466 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 11:07:47.414466 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 11:07:47.414466 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:07:47.414466 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 526/1058] 10.222.0.44 () {56 vars in 1096 bytes} [Wed Jun 29 11:07:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 11:07:47.419903 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5933b82d-54cc-48c1-a800-e4c81d538b21 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:47.468483 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9a4f6f-927e-46c6-860c-4cccb8943ea1 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:47.499915 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:47.505698 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e33d6b2-5ecd-41bc-94d7-4d9e37a002b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:47.506489 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e33d6b2-5ecd-41bc-94d7-4d9e37a002b2 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:47.506769 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e33d6b2-5ecd-41bc-94d7-4d9e37a002b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:47.507037 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e33d6b2-5ecd-41bc-94d7-4d9e37a002b2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:47.517359 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e33d6b2-5ecd-41bc-94d7-4d9e37a002b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:47.518832 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e33d6b2-5ecd-41bc-94d7-4d9e37a002b2 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:47.523861 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:47.523981 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5933b82d-54cc-48c1-a800-e4c81d538b21 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:47.537032 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-1e33d6b2-5ecd-41bc-94d7-4d9e37a002b2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 11:07:47.537032 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:07:47.537032 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:07:47.537032 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:07:47.537032 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:07:47.537032 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:07:47.537032 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:07:47.537032 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:07:47.537032 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:07:47.537032 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:07:47.537032 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:07:47.537032 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:07:47.537032 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 11:07:47.537032 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 11:07:47.537032 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 11:07:47.543277 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 11:07:47.543277 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:07:47.543277 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:07:47.543277 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:07:47.543277 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:07:47.543277 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:07:47.543277 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:07:47.543277 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:07:47.543277 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jun 29 11:07:47.543277 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:07:47.543277 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:07:47.543277 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:07:47.543277 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:07:47.543277 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:07:47.543277 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:07:47.543277 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:07:47.543277 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:07:47.544215 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 11:07:47.544215 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 11:07:47.544215 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 11:07:47.544215 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jun 29 11:07:47.544215 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 533/1059] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jun 29 11:07:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 11:07:47.544215 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27062a53-ea73-4cbb-a6aa-e6ef19c89760 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:47.544215 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27062a53-ea73-4cbb-a6aa-e6ef19c89760 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:47.544215 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27062a53-ea73-4cbb-a6aa-e6ef19c89760 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:47.544215 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 534/1060] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:07:47 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:07:47.544748 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27062a53-ea73-4cbb-a6aa-e6ef19c89760 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:47.568001 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7374725-ec4a-4820-ba8d-04b7c0805790 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:47.568668 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7374725-ec4a-4820-ba8d-04b7c0805790 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:47.568945 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7374725-ec4a-4820-ba8d-04b7c0805790 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:47.569218 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7374725-ec4a-4820-ba8d-04b7c0805790 None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:47.569955 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7374725-ec4a-4820-ba8d-04b7c0805790 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:07:47.570258 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7374725-ec4a-4820-ba8d-04b7c0805790 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:47.571922 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7374725-ec4a-4820-ba8d-04b7c0805790 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:47.579933 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 527/1061] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Jun 29 11:07:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 11:07:47.581119 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e44007-62a3-463d-9009-012f2eb22822 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:47.581402 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e44007-62a3-463d-9009-012f2eb22822 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:47.581673 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e44007-62a3-463d-9009-012f2eb22822 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:47.596925 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-85e44007-62a3-463d-9009-012f2eb22822 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:07:47.598107 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-85e44007-62a3-463d-9009-012f2eb22822 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:07:47.653092 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27062a53-ea73-4cbb-a6aa-e6ef19c89760 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:47.662894 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 528/1062] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:07:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 11:07:47.663884 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e44007-62a3-463d-9009-012f2eb22822 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:47.686871 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7021ed58-3664-4a3c-b9c3-9eccf39a4bb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:47.687523 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7021ed58-3664-4a3c-b9c3-9eccf39a4bb6 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:47.687804 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7021ed58-3664-4a3c-b9c3-9eccf39a4bb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:47.688110 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7021ed58-3664-4a3c-b9c3-9eccf39a4bb6 None admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:47.688820 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7021ed58-3664-4a3c-b9c3-9eccf39a4bb6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:07:47.689119 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7021ed58-3664-4a3c-b9c3-9eccf39a4bb6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:47.691113 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7021ed58-3664-4a3c-b9c3-9eccf39a4bb6 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:47.702345 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 535/1063] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Jun 29 11:07:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:07:47.703118 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7021ed58-3664-4a3c-b9c3-9eccf39a4bb6 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:47.733011 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:47.758323 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:47.765273 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e44007-62a3-463d-9009-012f2eb22822 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:47.797940 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a6caae-1a89-4330-9d8e-fca3ccf80f9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:47.798844 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a6caae-1a89-4330-9d8e-fca3ccf80f9e None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:47.799637 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7021ed58-3664-4a3c-b9c3-9eccf39a4bb6 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:47.799750 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a6caae-1a89-4330-9d8e-fca3ccf80f9e None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:47.799956 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a6caae-1a89-4330-9d8e-fca3ccf80f9e None admin] PATH_INFO: `/v3/users/d90e84e759f44197b69e8d41e7dbdff5` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:47.812910 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a6caae-1a89-4330-9d8e-fca3ccf80f9e None admin] RBAC: Authorizing `identity:get_user(user_id=d90e84e759f44197b69e8d41e7dbdff5)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:47.814334 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a6caae-1a89-4330-9d8e-fca3ccf80f9e None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:47.816766 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 529/1064] 10.222.0.44 () {56 vars in 1177 bytes} [Wed Jun 29 11:07:47 2022] GET /identity/v3/users/d90e84e759f44197b69e8d41e7dbdff5 => generated 285 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:07:47.817351 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a6caae-1a89-4330-9d8e-fca3ccf80f9e None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:47.838056 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7eeb70-af64-4ff2-8c4a-16376c51dfca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:47.838695 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7eeb70-af64-4ff2-8c4a-16376c51dfca None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:47.838974 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7eeb70-af64-4ff2-8c4a-16376c51dfca None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:47.839252 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7eeb70-af64-4ff2-8c4a-16376c51dfca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:47.904495 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7eeb70-af64-4ff2-8c4a-16376c51dfca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:47.905933 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7eeb70-af64-4ff2-8c4a-16376c51dfca None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:47.925164 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a6caae-1a89-4330-9d8e-fca3ccf80f9e None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:47.950155 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:47.958211 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecccbe1-13a0-47cf-8f6c-97707f993483 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:47.958990 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecccbe1-13a0-47cf-8f6c-97707f993483 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:47.959329 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecccbe1-13a0-47cf-8f6c-97707f993483 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:47.959599 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecccbe1-13a0-47cf-8f6c-97707f993483 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:47.960515 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:47.966469 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecccbe1-13a0-47cf-8f6c-97707f993483 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:47.967907 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecccbe1-13a0-47cf-8f6c-97707f993483 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:47.983214 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-cecccbe1-13a0-47cf-8f6c-97707f993483 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 29 11:07:47.983214 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:07:47.983214 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:07:47.983214 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:07:47.983214 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:07:47.983214 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:07:47.983214 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:07:47.983214 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:07:47.983214 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:07:47.983214 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:07:47.983214 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:07:47.983214 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:07:47.983214 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 11:07:47.983214 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 11:07:47.983214 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 11:07:47.983214 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 11:07:47.990544 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:07:47.990544 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:07:47.990544 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 11:07:47.990544 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 11:07:47.990544 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 11:07:47.990544 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 11:07:47.990544 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:07:47.990544 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:07:47.990544 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:07:47.990544 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:07:47.990544 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:07:47.990544 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jun 29 11:07:47.990544 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:07:47.990544 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:07:47.990544 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:07:47.990544 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:07:47.990544 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:07:47.990544 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:07:47.991519 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:07:47.991519 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:07:47.991519 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 11:07:47.991519 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 11:07:47.991519 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 11:07:47.991519 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 11:07:47.991519 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 11:07:47.991519 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 11:07:47.991519 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 29 11:07:47.991519 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:07:47.991519 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 530/1065] 10.222.0.44 () {56 vars in 1093 bytes} [Wed Jun 29 11:07:47 2022] GET /identity/v3/users/nova => generated 82 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:07:47.991519 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecccbe1-13a0-47cf-8f6c-97707f993483 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:48.006144 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 536/1066] 10.222.0.44 () {58 vars in 1282 bytes} [Wed Jun 29 11:07:47 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 29 11:07:48.007893 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7eeb70-af64-4ff2-8c4a-16376c51dfca None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:48.083061 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecccbe1-13a0-47cf-8f6c-97707f993483 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:48.102972 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7eeb70-af64-4ff2-8c4a-16376c51dfca None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:48.117614 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe63c3c-4cf2-4cbd-9070-1fa534ef3732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:48.118262 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe63c3c-4cf2-4cbd-9070-1fa534ef3732 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:48.118544 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe63c3c-4cf2-4cbd-9070-1fa534ef3732 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:48.118823 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe63c3c-4cf2-4cbd-9070-1fa534ef3732 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:48.136985 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47761a8b-3362-4dc4-b47b-4247346cc41e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:48.137638 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47761a8b-3362-4dc4-b47b-4247346cc41e None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:48.137915 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47761a8b-3362-4dc4-b47b-4247346cc41e None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:48.138185 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47761a8b-3362-4dc4-b47b-4247346cc41e None admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:48.139002 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47761a8b-3362-4dc4-b47b-4247346cc41e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:07:48.139312 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47761a8b-3362-4dc4-b47b-4247346cc41e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:48.140764 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47761a8b-3362-4dc4-b47b-4247346cc41e None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:48.157337 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 537/1067] 10.222.0.44 () {56 vars in 1133 bytes} [Wed Jun 29 11:07:48 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:07:48.158112 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47761a8b-3362-4dc4-b47b-4247346cc41e None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:48.173828 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:48.176410 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:48.188944 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe63c3c-4cf2-4cbd-9070-1fa534ef3732 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:48.190928 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe63c3c-4cf2-4cbd-9070-1fa534ef3732 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:48.257529 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47761a8b-3362-4dc4-b47b-4247346cc41e None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:48.286897 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 531/1068] 10.222.0.44 () {58 vars in 1282 bytes} [Wed Jun 29 11:07:47 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 29 11:07:48.287720 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe63c3c-4cf2-4cbd-9070-1fa534ef3732 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:48.291121 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a1ed11-5459-4617-ad3a-9dcc5c982c60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:48.291929 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a1ed11-5459-4617-ad3a-9dcc5c982c60 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:48.292214 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a1ed11-5459-4617-ad3a-9dcc5c982c60 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:48.292485 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a1ed11-5459-4617-ad3a-9dcc5c982c60 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:48.302351 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a1ed11-5459-4617-ad3a-9dcc5c982c60 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:48.303781 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a1ed11-5459-4617-ad3a-9dcc5c982c60 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:48.320674 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-23a1ed11-5459-4617-ad3a-9dcc5c982c60 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 11:07:48.320674 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:07:48.320674 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:07:48.320674 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:07:48.320674 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:07:48.320674 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:07:48.320674 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:07:48.320674 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:07:48.320674 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:07:48.320674 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:07:48.320674 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:07:48.320674 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:07:48.320674 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 11:07:48.320674 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 11:07:48.320674 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 11:07:48.321849 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 11:07:48.321849 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:07:48.321849 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:07:48.321849 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:07:48.321849 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:07:48.321849 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:07:48.321849 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:07:48.321849 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:07:48.321849 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jun 29 11:07:48.321849 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:07:48.321849 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:07:48.321849 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:07:48.321849 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:07:48.321849 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:07:48.321849 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:07:48.321849 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:07:48.321849 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:07:48.322776 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 11:07:48.322776 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 11:07:48.322776 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 11:07:48.322776 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jun 29 11:07:48.322776 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 538/1069] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jun 29 11:07:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:07:48.323329 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a1ed11-5459-4617-ad3a-9dcc5c982c60 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:48.378057 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:48.386629 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:48.393555 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe63c3c-4cf2-4cbd-9070-1fa534ef3732 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:48.420241 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a1ed11-5459-4617-ad3a-9dcc5c982c60 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:48.427230 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007ef3bd-cd10-494f-be02-e5cc53a07cbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:48.427980 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007ef3bd-cd10-494f-be02-e5cc53a07cbe None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:48.428261 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007ef3bd-cd10-494f-be02-e5cc53a07cbe None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:48.428532 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007ef3bd-cd10-494f-be02-e5cc53a07cbe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:48.438338 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007ef3bd-cd10-494f-be02-e5cc53a07cbe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:48.439796 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007ef3bd-cd10-494f-be02-e5cc53a07cbe None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:48.454375 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9961eb-f4cb-4e7f-9655-7a4606a673a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:48.455220 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9961eb-f4cb-4e7f-9655-7a4606a673a6 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:48.456035 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9961eb-f4cb-4e7f-9655-7a4606a673a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:48.456313 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9961eb-f4cb-4e7f-9655-7a4606a673a6 None admin] PATH_INFO: `/v3/projects/31fa73283b1a4cffae7754d453ae4689` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:48.457489 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9961eb-f4cb-4e7f-9655-7a4606a673a6 None admin] RBAC: Authorizing `identity:get_project(project_id=31fa73283b1a4cffae7754d453ae4689)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:48.457961 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-007ef3bd-cd10-494f-be02-e5cc53a07cbe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 11:07:48.457961 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:07:48.457961 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:07:48.457961 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:07:48.457961 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:07:48.457961 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:07:48.457961 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:07:48.457961 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:07:48.457961 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:07:48.457961 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:07:48.457961 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:07:48.457961 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:07:48.457961 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 11:07:48.457961 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 11:07:48.457961 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 11:07:48.458973 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 11:07:48.458973 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:07:48.458973 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:07:48.458973 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:07:48.458973 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:07:48.458973 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:07:48.458973 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:07:48.458973 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:07:48.458973 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jun 29 11:07:48.458973 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:07:48.458973 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:07:48.458973 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:07:48.458973 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:07:48.458973 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:07:48.458973 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:07:48.458973 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:07:48.458973 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:07:48.459954 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 11:07:48.459954 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 11:07:48.459954 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 11:07:48.459954 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:07:48.460177 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9961eb-f4cb-4e7f-9655-7a4606a673a6 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:48.460246 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 532/1070] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jun 29 11:07:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:07:48.460246 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007ef3bd-cd10-494f-be02-e5cc53a07cbe None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:48.465592 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 539/1071] 10.222.0.44 () {56 vars in 1186 bytes} [Wed Jun 29 11:07:48 2022] GET /identity/v3/projects/31fa73283b1a4cffae7754d453ae4689 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:07:48.465592 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9961eb-f4cb-4e7f-9655-7a4606a673a6 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:48.559193 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007ef3bd-cd10-494f-be02-e5cc53a07cbe None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:48.573400 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9961eb-f4cb-4e7f-9655-7a4606a673a6 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:48.597014 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768241d2-23d7-4aac-b1db-50b5fdcf0605 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:48.597675 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768241d2-23d7-4aac-b1db-50b5fdcf0605 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:48.597959 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768241d2-23d7-4aac-b1db-50b5fdcf0605 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:48.598234 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768241d2-23d7-4aac-b1db-50b5fdcf0605 None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:48.598940 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768241d2-23d7-4aac-b1db-50b5fdcf0605 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:07:48.599248 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768241d2-23d7-4aac-b1db-50b5fdcf0605 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:48.600628 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768241d2-23d7-4aac-b1db-50b5fdcf0605 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:48.606633 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94e9879-166b-4459-869e-eb9d945dab14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:48.607062 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:48.607319 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94e9879-166b-4459-869e-eb9d945dab14 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:48.608010 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94e9879-166b-4459-869e-eb9d945dab14 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:48.608289 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94e9879-166b-4459-869e-eb9d945dab14 None admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:48.608999 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94e9879-166b-4459-869e-eb9d945dab14 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:07:48.609301 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94e9879-166b-4459-869e-eb9d945dab14 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:48.610679 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94e9879-166b-4459-869e-eb9d945dab14 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:48.612113 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:48.612473 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 533/1072] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Jun 29 11:07:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:07:48.623897 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768241d2-23d7-4aac-b1db-50b5fdcf0605 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:48.640485 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 540/1073] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Jun 29 11:07:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:07:48.642242 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94e9879-166b-4459-869e-eb9d945dab14 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:48.723801 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768241d2-23d7-4aac-b1db-50b5fdcf0605 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:48.758921 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8c9d5e-0722-4171-bf35-8b7d96030465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:48.759582 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8c9d5e-0722-4171-bf35-8b7d96030465 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:48.759890 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8c9d5e-0722-4171-bf35-8b7d96030465 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:48.760172 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8c9d5e-0722-4171-bf35-8b7d96030465 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:48.761006 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8c9d5e-0722-4171-bf35-8b7d96030465 None admin] RBAC: Adding query filter params (role.id=e95c2ce63e114cc19a9dda6ad21e794f, scope.project.id=31fa73283b1a4cffae7754d453ae4689, user.id=d90e84e759f44197b69e8d41e7dbdff5) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:07:48.761313 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8c9d5e-0722-4171-bf35-8b7d96030465 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:48.762331 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94e9879-166b-4459-869e-eb9d945dab14 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:48.762739 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8c9d5e-0722-4171-bf35-8b7d96030465 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:48.776270 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 534/1074] 10.222.0.44 () {56 vars in 1374 bytes} [Wed Jun 29 11:07:48 2022] GET /identity/v3/role_assignments?user.id=d90e84e759f44197b69e8d41e7dbdff5&scope.project.id=31fa73283b1a4cffae7754d453ae4689&role.id=e95c2ce63e114cc19a9dda6ad21e794f => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:07:48.799603 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6e415b-f511-4599-8616-19a6a3542877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:48.800467 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6e415b-f511-4599-8616-19a6a3542877 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:48.800750 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6e415b-f511-4599-8616-19a6a3542877 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:48.801023 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6e415b-f511-4599-8616-19a6a3542877 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:48.808725 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6e415b-f511-4599-8616-19a6a3542877 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:48.810199 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6e415b-f511-4599-8616-19a6a3542877 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:48.826593 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-8f6e415b-f511-4599-8616-19a6a3542877 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 11:07:48.826593 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:07:48.826593 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:07:48.826593 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:07:48.826593 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:07:48.826593 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:07:48.826593 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:07:48.826593 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:07:48.826593 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:07:48.826593 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:07:48.826593 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:07:48.826593 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:07:48.826593 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 11:07:48.826593 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 11:07:48.826593 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 11:07:48.827571 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 11:07:48.827571 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:07:48.827571 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:07:48.827571 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:07:48.827571 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:07:48.827571 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:07:48.827571 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:07:48.827571 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:07:48.827571 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jun 29 11:07:48.827571 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:07:48.827571 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:07:48.827571 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:07:48.827571 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:07:48.827571 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:07:48.827571 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:07:48.827571 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:07:48.827571 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:07:48.828790 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 11:07:48.828790 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 11:07:48.828790 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 11:07:48.828790 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 11:07:48.828790 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 11:07:48.828790 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 11:07:48.828790 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 11:07:48.828790 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jun 29 11:07:48.832358 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 541/1075] 10.222.0.44 () {56 vars in 1111 bytes} [Wed Jun 29 11:07:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:07:48.834404 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8c9d5e-0722-4171-bf35-8b7d96030465 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:48.911762 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:48.958980 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8c9d5e-0722-4171-bf35-8b7d96030465 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:48.993618 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ac7cf6-9d6e-44b6-bfc0-bd5afa304df0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:48.994210 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ac7cf6-9d6e-44b6-bfc0-bd5afa304df0 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:48.994492 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ac7cf6-9d6e-44b6-bfc0-bd5afa304df0 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:48.994764 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ac7cf6-9d6e-44b6-bfc0-bd5afa304df0 None admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:48.995539 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ac7cf6-9d6e-44b6-bfc0-bd5afa304df0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:07:48.995870 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ac7cf6-9d6e-44b6-bfc0-bd5afa304df0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:48.997301 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ac7cf6-9d6e-44b6-bfc0-bd5afa304df0 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:49.010060 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 535/1076] 10.222.0.44 () {56 vars in 1148 bytes} [Wed Jun 29 11:07:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:07:49.016086 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6e415b-f511-4599-8616-19a6a3542877 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:49.070790 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:49.113799 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6e415b-f511-4599-8616-19a6a3542877 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:49.146461 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf7b814-f8ca-49f6-b072-4dfd0beb90c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:49.147327 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf7b814-f8ca-49f6-b072-4dfd0beb90c1 None admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:49.147605 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf7b814-f8ca-49f6-b072-4dfd0beb90c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:49.147899 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf7b814-f8ca-49f6-b072-4dfd0beb90c1 None admin] PATH_INFO: `/v3/projects/ebe5dffa4bb8440286687daefb6c98f4/users/796aeeff16df4739bddec1f6dee61bfe/roles/3e8ef91b147f4b6b803ef07f3301be3f` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:49.150058 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf7b814-f8ca-49f6-b072-4dfd0beb90c1 None admin] RBAC: Authorizing `identity:create_grant(project_id=ebe5dffa4bb8440286687daefb6c98f4, user_id=796aeeff16df4739bddec1f6dee61bfe, role_id=3e8ef91b147f4b6b803ef07f3301be3f)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:49.151477 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf7b814-f8ca-49f6-b072-4dfd0beb90c1 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:49.242730 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 542/1077] 10.222.0.44 () {58 vars in 1439 bytes} [Wed Jun 29 11:07:49 2022] PUT /identity/v3/projects/ebe5dffa4bb8440286687daefb6c98f4/users/796aeeff16df4739bddec1f6dee61bfe/roles/3e8ef91b147f4b6b803ef07f3301be3f => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:07:50.787186 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b830740-d4fe-404e-b732-c86fc219f0dd None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:50.787744 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b830740-d4fe-404e-b732-c86fc219f0dd None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:50.787744 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b830740-d4fe-404e-b732-c86fc219f0dd None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:50.789123 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 536/1078] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:07:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:07:50.794474 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff63032-1575-4121-95a9-b8ea502586a2 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:50.794869 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff63032-1575-4121-95a9-b8ea502586a2 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:50.795024 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff63032-1575-4121-95a9-b8ea502586a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:50.811298 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-2ff63032-1575-4121-95a9-b8ea502586a2 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:07:50.812498 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-2ff63032-1575-4121-95a9-b8ea502586a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:07:50.878385 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 543/1079] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:07:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 11:07:50.900888 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b830740-d4fe-404e-b732-c86fc219f0dd None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:51.001389 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b830740-d4fe-404e-b732-c86fc219f0dd None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:51.023917 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:51.035020 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d56ed52-d7ec-4f34-9038-9cb79779f72c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:51.035799 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d56ed52-d7ec-4f34-9038-9cb79779f72c None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:51.036105 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d56ed52-d7ec-4f34-9038-9cb79779f72c None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:51.049747 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d56ed52-d7ec-4f34-9038-9cb79779f72c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:51.049747 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d56ed52-d7ec-4f34-9038-9cb79779f72c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 11:07:51.053723 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-5d56ed52-d7ec-4f34-9038-9cb79779f72c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 11:07:51.053723 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:07:51.053723 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:07:51.053723 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:07:51.053723 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:07:51.053723 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:07:51.053723 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:07:51.053723 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:07:51.053723 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:07:51.053723 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:07:51.053723 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:07:51.053723 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:07:51.053723 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 11:07:51.053723 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 11:07:51.053723 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 11:07:51.053723 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 11:07:51.054860 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 11:07:51.054860 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 11:07:51.054860 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 11:07:51.054860 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 11:07:51.054860 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:07:51.054860 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:07:51.054860 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:07:51.054860 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:07:51.054860 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:07:51.054860 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:07:51.054860 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:07:51.054860 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jun 29 11:07:51.054860 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:07:51.054860 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:07:51.054860 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:07:51.054860 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:07:51.055876 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:07:51.055876 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:07:51.055876 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:07:51.055876 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:07:51.055876 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 11:07:51.055876 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 11:07:51.055876 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 11:07:51.055876 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 11:07:51.055876 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 11:07:51.055876 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 11:07:51.055876 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 11:07:51.055876 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:07:51.055876 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 537/1080] 10.222.0.44 () {56 vars in 1096 bytes} [Wed Jun 29 11:07:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:07:51.061022 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff63032-1575-4121-95a9-b8ea502586a2 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:51.106233 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:51.158200 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff63032-1575-4121-95a9-b8ea502586a2 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:51.191052 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50efd05e-f41d-454b-8b9a-bd69a16a63c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:51.191706 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50efd05e-f41d-454b-8b9a-bd69a16a63c5 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:51.192030 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50efd05e-f41d-454b-8b9a-bd69a16a63c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:51.192299 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50efd05e-f41d-454b-8b9a-bd69a16a63c5 None admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:51.193034 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50efd05e-f41d-454b-8b9a-bd69a16a63c5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:07:51.193333 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50efd05e-f41d-454b-8b9a-bd69a16a63c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:51.194684 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50efd05e-f41d-454b-8b9a-bd69a16a63c5 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:51.202736 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 544/1081] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Jun 29 11:07:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:07:51.209112 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d56ed52-d7ec-4f34-9038-9cb79779f72c None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:51.255818 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2659ac-da2d-42d4-b535-c24fbe708c1c None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:51.256132 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2659ac-da2d-42d4-b535-c24fbe708c1c None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:51.256421 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2659ac-da2d-42d4-b535-c24fbe708c1c None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:51.257779 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 545/1082] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:07:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 29 11:07:51.262441 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a569b5-f209-4099-aa24-17123006f195 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:51.262729 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a569b5-f209-4099-aa24-17123006f195 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:51.263006 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a569b5-f209-4099-aa24-17123006f195 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:51.279752 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-90a569b5-f209-4099-aa24-17123006f195 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:07:51.280997 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-90a569b5-f209-4099-aa24-17123006f195 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:07:51.307872 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d56ed52-d7ec-4f34-9038-9cb79779f72c None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:51.341968 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f5ec05-573c-4886-9623-8434ea970765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:51.342831 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f5ec05-573c-4886-9623-8434ea970765 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:51.343113 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f5ec05-573c-4886-9623-8434ea970765 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:51.343396 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f5ec05-573c-4886-9623-8434ea970765 None admin] PATH_INFO: `/v3/users/d90e84e759f44197b69e8d41e7dbdff5` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:51.344604 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f5ec05-573c-4886-9623-8434ea970765 None admin] RBAC: Authorizing `identity:get_user(user_id=d90e84e759f44197b69e8d41e7dbdff5)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:51.346022 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f5ec05-573c-4886-9623-8434ea970765 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:51.347761 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 546/1083] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:07:51 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 29 11:07:51.347901 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 538/1084] 10.222.0.44 () {56 vars in 1177 bytes} [Wed Jun 29 11:07:51 2022] GET /identity/v3/users/d90e84e759f44197b69e8d41e7dbdff5 => generated 285 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:07:51.353729 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a569b5-f209-4099-aa24-17123006f195 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:51.370874 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f5ec05-573c-4886-9623-8434ea970765 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:51.387951 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:51.418860 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:51.453362 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a569b5-f209-4099-aa24-17123006f195 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:51.468196 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f5ec05-573c-4886-9623-8434ea970765 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:51.486388 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73fadb8-3996-43ea-b843-73d888520ffd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:51.487771 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73fadb8-3996-43ea-b843-73d888520ffd None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:51.488089 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73fadb8-3996-43ea-b843-73d888520ffd None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:51.488368 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73fadb8-3996-43ea-b843-73d888520ffd None admin] PATH_INFO: `/v3/projects/31fa73283b1a4cffae7754d453ae4689` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:51.489560 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73fadb8-3996-43ea-b843-73d888520ffd None admin] RBAC: Authorizing `identity:get_project(project_id=31fa73283b1a4cffae7754d453ae4689)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:51.490991 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73fadb8-3996-43ea-b843-73d888520ffd None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:51.492732 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 547/1085] 10.222.0.44 () {56 vars in 1186 bytes} [Wed Jun 29 11:07:51 2022] GET /identity/v3/projects/31fa73283b1a4cffae7754d453ae4689 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:07:51.498562 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73fadb8-3996-43ea-b843-73d888520ffd None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:51.501730 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df12528e-6569-4bfb-ba69-cd14fff4b200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:51.503097 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df12528e-6569-4bfb-ba69-cd14fff4b200 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:51.503374 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df12528e-6569-4bfb-ba69-cd14fff4b200 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:51.503644 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df12528e-6569-4bfb-ba69-cd14fff4b200 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:51.514205 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df12528e-6569-4bfb-ba69-cd14fff4b200 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 11:07:51.522343 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-df12528e-6569-4bfb-ba69-cd14fff4b200 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 29 11:07:51.522343 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:07:51.522343 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:07:51.522343 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:07:51.522343 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:07:51.522343 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:07:51.522343 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:07:51.522343 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:07:51.522343 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:07:51.522343 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:07:51.522343 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:07:51.522343 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:07:51.522343 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 11:07:51.522343 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 11:07:51.522343 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 11:07:51.522343 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 11:07:51.523336 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 11:07:51.523336 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 11:07:51.523336 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 11:07:51.523336 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 11:07:51.523336 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:07:51.523336 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:07:51.523336 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:07:51.523336 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:07:51.523336 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:07:51.523336 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:07:51.523336 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:07:51.523336 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jun 29 11:07:51.523336 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:07:51.523336 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:07:51.523336 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:07:51.523336 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:07:51.524604 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:07:51.524604 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:07:51.524604 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:07:51.524604 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:07:51.524604 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 11:07:51.524604 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 11:07:51.524604 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 11:07:51.524604 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 11:07:51.524604 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 11:07:51.524604 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 11:07:51.524604 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 29 11:07:51.524604 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:07:51.524604 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 539/1086] 10.222.0.44 () {56 vars in 1120 bytes} [Wed Jun 29 11:07:51 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 11:07:51.529265 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df12528e-6569-4bfb-ba69-cd14fff4b200 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:51.647665 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73fadb8-3996-43ea-b843-73d888520ffd None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:51.649911 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:51.678017 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df12528e-6569-4bfb-ba69-cd14fff4b200 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:51.683188 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da280901-00a9-43e5-b022-c44023bb24cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:51.684107 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da280901-00a9-43e5-b022-c44023bb24cf None admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:51.684390 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da280901-00a9-43e5-b022-c44023bb24cf None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:51.684671 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da280901-00a9-43e5-b022-c44023bb24cf None admin] PATH_INFO: `/v3/projects/31fa73283b1a4cffae7754d453ae4689/users/d90e84e759f44197b69e8d41e7dbdff5/roles/e95c2ce63e114cc19a9dda6ad21e794f` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:51.686663 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da280901-00a9-43e5-b022-c44023bb24cf None admin] RBAC: Authorizing `identity:create_grant(project_id=31fa73283b1a4cffae7754d453ae4689, user_id=d90e84e759f44197b69e8d41e7dbdff5, role_id=e95c2ce63e114cc19a9dda6ad21e794f)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:51.687986 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:51.688146 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da280901-00a9-43e5-b022-c44023bb24cf None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:51.710297 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e030b769-cb38-4343-b89f-6ab289a28dcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:51.710951 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e030b769-cb38-4343-b89f-6ab289a28dcd None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:51.711230 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e030b769-cb38-4343-b89f-6ab289a28dcd None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:51.711501 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e030b769-cb38-4343-b89f-6ab289a28dcd None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:51.712263 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e030b769-cb38-4343-b89f-6ab289a28dcd None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:07:51.712566 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e030b769-cb38-4343-b89f-6ab289a28dcd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:51.713886 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e030b769-cb38-4343-b89f-6ab289a28dcd None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:51.722834 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 540/1087] 10.222.0.44 () {56 vars in 1115 bytes} [Wed Jun 29 11:07:51 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 11:07:51.728493 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e030b769-cb38-4343-b89f-6ab289a28dcd None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:51.744481 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 548/1088] 10.222.0.44 () {58 vars in 1439 bytes} [Wed Jun 29 11:07:51 2022] PUT /identity/v3/projects/31fa73283b1a4cffae7754d453ae4689/users/d90e84e759f44197b69e8d41e7dbdff5/roles/e95c2ce63e114cc19a9dda6ad21e794f => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:07:51.864653 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e030b769-cb38-4343-b89f-6ab289a28dcd None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:51.898748 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc628555-52cf-401f-b6fd-5f6ccb879ca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:51.899463 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc628555-52cf-401f-b6fd-5f6ccb879ca3 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:51.899740 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc628555-52cf-401f-b6fd-5f6ccb879ca3 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:51.900040 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc628555-52cf-401f-b6fd-5f6ccb879ca3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:51.914687 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc628555-52cf-401f-b6fd-5f6ccb879ca3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:51.916161 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc628555-52cf-401f-b6fd-5f6ccb879ca3 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:51.935081 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-cc628555-52cf-401f-b6fd-5f6ccb879ca3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 11:07:51.935081 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:07:51.935081 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:07:51.935081 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:07:51.935081 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:07:51.935081 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:07:51.935081 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:07:51.935081 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:07:51.935081 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:07:51.935081 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:07:51.935081 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:07:51.935081 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:07:51.935081 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 11:07:51.935081 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 11:07:51.935081 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 11:07:51.936179 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 11:07:51.936179 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:07:51.936179 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:07:51.936179 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:07:51.936179 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:07:51.936179 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:07:51.936179 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:07:51.936179 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:07:51.936179 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jun 29 11:07:51.936179 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:07:51.936179 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:07:51.936179 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:07:51.936179 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:07:51.936179 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:07:51.936179 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:07:51.936179 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:07:51.936179 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:07:51.937136 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 11:07:51.937136 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 11:07:51.937136 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 11:07:51.937136 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:07:51.937136 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 541/1089] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jun 29 11:07:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 11:07:51.942198 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da280901-00a9-43e5-b022-c44023bb24cf None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:52.041586 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da280901-00a9-43e5-b022-c44023bb24cf None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:52.074690 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b02fc6b-6a33-43ac-b7fb-955bb4b39f90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:52.075352 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b02fc6b-6a33-43ac-b7fb-955bb4b39f90 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:52.075639 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b02fc6b-6a33-43ac-b7fb-955bb4b39f90 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:52.075952 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b02fc6b-6a33-43ac-b7fb-955bb4b39f90 None admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:52.076670 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b02fc6b-6a33-43ac-b7fb-955bb4b39f90 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:07:52.076978 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b02fc6b-6a33-43ac-b7fb-955bb4b39f90 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:52.078343 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b02fc6b-6a33-43ac-b7fb-955bb4b39f90 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:52.090034 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 549/1090] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Jun 29 11:07:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:07:52.095510 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc628555-52cf-401f-b6fd-5f6ccb879ca3 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:52.121157 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:52.193206 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc628555-52cf-401f-b6fd-5f6ccb879ca3 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:52.231407 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e653271-d13f-4089-987a-c8ca4297ae38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:52.232225 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e653271-d13f-4089-987a-c8ca4297ae38 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:52.232504 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e653271-d13f-4089-987a-c8ca4297ae38 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:52.232779 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e653271-d13f-4089-987a-c8ca4297ae38 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:52.240306 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e653271-d13f-4089-987a-c8ca4297ae38 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:52.241726 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e653271-d13f-4089-987a-c8ca4297ae38 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:52.255506 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-2e653271-d13f-4089-987a-c8ca4297ae38 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 29 11:07:52.255506 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:07:52.255506 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:07:52.255506 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:07:52.255506 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:07:52.255506 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:07:52.255506 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:07:52.255506 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:07:52.255506 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:07:52.255506 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:07:52.255506 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:07:52.255506 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:07:52.255506 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 11:07:52.255506 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 11:07:52.255506 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 11:07:52.255506 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 11:07:52.256574 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:07:52.256574 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:07:52.256574 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 11:07:52.256574 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 11:07:52.256574 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 11:07:52.256574 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 11:07:52.256574 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:07:52.256574 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:07:52.256574 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:07:52.256574 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:07:52.256574 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:07:52.256574 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jun 29 11:07:52.256574 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:07:52.256574 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:07:52.256574 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:07:52.256574 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:07:52.256574 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:07:52.256574 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:07:52.257528 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:07:52.257528 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:07:52.257528 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 11:07:52.257528 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 11:07:52.257528 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 11:07:52.257528 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 11:07:52.257528 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 11:07:52.257528 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 11:07:52.257528 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 29 11:07:52.257528 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:07:52.257528 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 542/1091] 10.222.0.44 () {56 vars in 1093 bytes} [Wed Jun 29 11:07:52 2022] GET /identity/v3/users/nova => generated 82 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 11:07:52.262380 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b02fc6b-6a33-43ac-b7fb-955bb4b39f90 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:52.293107 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:52.365826 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b02fc6b-6a33-43ac-b7fb-955bb4b39f90 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:52.399323 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c533ab-ea8c-431e-b834-1e0bd6314d7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:52.400014 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c533ab-ea8c-431e-b834-1e0bd6314d7b None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:52.400299 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c533ab-ea8c-431e-b834-1e0bd6314d7b None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:52.400571 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c533ab-ea8c-431e-b834-1e0bd6314d7b None admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:52.401405 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c533ab-ea8c-431e-b834-1e0bd6314d7b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:07:52.401720 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c533ab-ea8c-431e-b834-1e0bd6314d7b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:52.403143 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c533ab-ea8c-431e-b834-1e0bd6314d7b None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:52.420391 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 550/1092] 10.222.0.44 () {56 vars in 1133 bytes} [Wed Jun 29 11:07:52 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:07:52.425994 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e653271-d13f-4089-987a-c8ca4297ae38 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:52.509751 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:52.525851 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e653271-d13f-4089-987a-c8ca4297ae38 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:52.563787 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee30229-fc38-4b5e-a837-6c6d86a6db68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:52.564464 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee30229-fc38-4b5e-a837-6c6d86a6db68 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:52.565259 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee30229-fc38-4b5e-a837-6c6d86a6db68 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:52.565532 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee30229-fc38-4b5e-a837-6c6d86a6db68 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:52.635028 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee30229-fc38-4b5e-a837-6c6d86a6db68 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:52.636521 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee30229-fc38-4b5e-a837-6c6d86a6db68 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:52.715278 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:52.731016 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 543/1093] 10.222.0.44 () {58 vars in 1282 bytes} [Wed Jun 29 11:07:52 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 29 11:07:52.737200 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c533ab-ea8c-431e-b834-1e0bd6314d7b None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:52.814639 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:52.832928 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c533ab-ea8c-431e-b834-1e0bd6314d7b None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:52.867477 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6663f9-05f6-4108-9ad9-67eaad66b2d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:52.868304 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6663f9-05f6-4108-9ad9-67eaad66b2d7 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:52.868588 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6663f9-05f6-4108-9ad9-67eaad66b2d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:52.868859 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6663f9-05f6-4108-9ad9-67eaad66b2d7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:52.878917 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6663f9-05f6-4108-9ad9-67eaad66b2d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:52.880385 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6663f9-05f6-4108-9ad9-67eaad66b2d7 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:52.897655 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-fb6663f9-05f6-4108-9ad9-67eaad66b2d7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 11:07:52.897655 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:07:52.897655 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:07:52.897655 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:07:52.897655 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:07:52.897655 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:07:52.897655 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:07:52.897655 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:07:52.897655 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:07:52.897655 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:07:52.897655 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:07:52.897655 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:07:52.897655 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 11:07:52.897655 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 11:07:52.897655 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 11:07:52.898732 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 11:07:52.898732 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:07:52.898732 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:07:52.898732 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:07:52.898732 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:07:52.898732 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:07:52.898732 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:07:52.898732 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:07:52.898732 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jun 29 11:07:52.898732 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:07:52.898732 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:07:52.898732 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:07:52.898732 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:07:52.898732 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:07:52.898732 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:07:52.898732 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:07:52.898732 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:07:52.904364 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 11:07:52.904364 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 11:07:52.904364 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 11:07:52.904364 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jun 29 11:07:52.904364 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 551/1094] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jun 29 11:07:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:07:52.904691 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee30229-fc38-4b5e-a837-6c6d86a6db68 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:53.000347 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee30229-fc38-4b5e-a837-6c6d86a6db68 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:53.032752 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567af0c7-be0b-4cbc-a394-9fe9be7014ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:53.034016 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567af0c7-be0b-4cbc-a394-9fe9be7014ac None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:53.034300 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567af0c7-be0b-4cbc-a394-9fe9be7014ac None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:53.034577 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567af0c7-be0b-4cbc-a394-9fe9be7014ac None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:53.035350 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567af0c7-be0b-4cbc-a394-9fe9be7014ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:07:53.035658 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567af0c7-be0b-4cbc-a394-9fe9be7014ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:53.037085 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567af0c7-be0b-4cbc-a394-9fe9be7014ac None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:53.047977 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 544/1095] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Jun 29 11:07:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:07:53.053527 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6663f9-05f6-4108-9ad9-67eaad66b2d7 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:53.149579 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6663f9-05f6-4108-9ad9-67eaad66b2d7 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:53.181405 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65334762-2698-49bc-945f-84a43ef648be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:53.182178 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65334762-2698-49bc-945f-84a43ef648be None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:53.182456 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65334762-2698-49bc-945f-84a43ef648be None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:53.182724 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65334762-2698-49bc-945f-84a43ef648be None admin] PATH_INFO: `/v3/projects/service` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:53.184601 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:53.190217 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65334762-2698-49bc-945f-84a43ef648be None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:53.191638 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65334762-2698-49bc-945f-84a43ef648be None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:53.206304 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-65334762-2698-49bc-945f-84a43ef648be None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 11:07:53.206304 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:07:53.206304 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:07:53.206304 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:07:53.206304 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:07:53.206304 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:07:53.206304 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:07:53.206304 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:07:53.206304 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:07:53.206304 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:07:53.206304 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:07:53.206304 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:07:53.206304 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 11:07:53.206304 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 11:07:53.206304 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 11:07:53.207214 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 11:07:53.207214 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:07:53.207214 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:07:53.207214 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:07:53.207214 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:07:53.207214 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:07:53.207214 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:07:53.207214 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:07:53.207214 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jun 29 11:07:53.207214 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:07:53.207214 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:07:53.207214 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:07:53.207214 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:07:53.207214 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:07:53.207214 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:07:53.207214 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:07:53.207214 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:07:53.208156 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 11:07:53.208156 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 11:07:53.208156 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 11:07:53.208156 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 11:07:53.208156 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 11:07:53.208156 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 11:07:53.208156 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 11:07:53.208156 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jun 29 11:07:53.208156 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 552/1096] 10.222.0.44 () {56 vars in 1111 bytes} [Wed Jun 29 11:07:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 11:07:53.213311 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567af0c7-be0b-4cbc-a394-9fe9be7014ac None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:53.258538 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:53.311540 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567af0c7-be0b-4cbc-a394-9fe9be7014ac None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:53.345741 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f438d1-00c6-4184-9c6f-de651947d47f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:53.346422 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f438d1-00c6-4184-9c6f-de651947d47f None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:53.346708 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f438d1-00c6-4184-9c6f-de651947d47f None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:53.346988 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f438d1-00c6-4184-9c6f-de651947d47f None admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:53.347769 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f438d1-00c6-4184-9c6f-de651947d47f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:07:53.348112 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f438d1-00c6-4184-9c6f-de651947d47f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:53.349527 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f438d1-00c6-4184-9c6f-de651947d47f None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:53.361723 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 545/1097] 10.222.0.44 () {56 vars in 1148 bytes} [Wed Jun 29 11:07:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:07:53.367505 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65334762-2698-49bc-945f-84a43ef648be None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:53.468219 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65334762-2698-49bc-945f-84a43ef648be None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:53.502765 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5aff435-8fef-4d0b-b40e-e55790259a6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:53.503414 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5aff435-8fef-4d0b-b40e-e55790259a6c None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:53.503694 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5aff435-8fef-4d0b-b40e-e55790259a6c None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:53.504002 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5aff435-8fef-4d0b-b40e-e55790259a6c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:53.504824 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5aff435-8fef-4d0b-b40e-e55790259a6c None admin] RBAC: Adding query filter params (role.id=3e8ef91b147f4b6b803ef07f3301be3f, scope.project.id=ebe5dffa4bb8440286687daefb6c98f4, user.id=796aeeff16df4739bddec1f6dee61bfe) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:07:53.505136 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5aff435-8fef-4d0b-b40e-e55790259a6c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:53.506565 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5aff435-8fef-4d0b-b40e-e55790259a6c None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:53.512254 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 553/1098] 10.222.0.44 () {56 vars in 1374 bytes} [Wed Jun 29 11:07:53 2022] GET /identity/v3/role_assignments?user.id=796aeeff16df4739bddec1f6dee61bfe&scope.project.id=ebe5dffa4bb8440286687daefb6c98f4&role.id=3e8ef91b147f4b6b803ef07f3301be3f => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:07:53.782715 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491cef7a-e819-4f6d-a4de-97902eff94fe None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:53.782932 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491cef7a-e819-4f6d-a4de-97902eff94fe None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:53.783207 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491cef7a-e819-4f6d-a4de-97902eff94fe None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:53.784666 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 546/1099] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:07:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:07:53.790068 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc98463-f171-4fe1-9dfb-e67187215faa None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:53.790354 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc98463-f171-4fe1-9dfb-e67187215faa None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:53.790628 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc98463-f171-4fe1-9dfb-e67187215faa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:53.808200 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-9cc98463-f171-4fe1-9dfb-e67187215faa None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:07:53.809391 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-9cc98463-f171-4fe1-9dfb-e67187215faa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:07:53.865520 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:53.879287 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 554/1100] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:07:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 11:07:53.903641 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491cef7a-e819-4f6d-a4de-97902eff94fe None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:54.010329 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491cef7a-e819-4f6d-a4de-97902eff94fe None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:54.012670 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:54.044520 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d537c3-866c-4fbf-afe7-90553779eaaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:54.045556 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d537c3-866c-4fbf-afe7-90553779eaaa None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:54.046063 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d537c3-866c-4fbf-afe7-90553779eaaa None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:54.046653 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d537c3-866c-4fbf-afe7-90553779eaaa None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:54.057769 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d537c3-866c-4fbf-afe7-90553779eaaa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 11:07:54.065627 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-61d537c3-866c-4fbf-afe7-90553779eaaa None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 11:07:54.065627 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:07:54.065627 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:07:54.065627 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:07:54.065627 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:07:54.065627 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:07:54.065627 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:07:54.065627 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:07:54.065627 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:07:54.065627 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:07:54.065627 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:07:54.065627 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:07:54.065627 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 11:07:54.065627 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 11:07:54.065627 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 11:07:54.065627 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 11:07:54.066618 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 11:07:54.066618 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 11:07:54.066618 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 11:07:54.066618 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 11:07:54.066618 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:07:54.066618 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:07:54.066618 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:07:54.066618 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:07:54.066618 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:07:54.066618 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:07:54.066618 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:07:54.066618 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jun 29 11:07:54.066618 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:07:54.066618 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:07:54.066618 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:07:54.066618 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:07:54.067467 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:07:54.067467 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:07:54.067467 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:07:54.067467 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:07:54.067467 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 11:07:54.067467 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 11:07:54.067467 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 11:07:54.067467 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 11:07:54.067467 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 11:07:54.067467 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 11:07:54.067467 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 11:07:54.067467 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:07:54.067467 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 547/1101] 10.222.0.44 () {56 vars in 1096 bytes} [Wed Jun 29 11:07:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:07:54.073377 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc98463-f171-4fe1-9dfb-e67187215faa None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:54.171738 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc98463-f171-4fe1-9dfb-e67187215faa None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:54.206820 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5e865a-f33c-45c7-9025-411edb9bfbf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:54.207488 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5e865a-f33c-45c7-9025-411edb9bfbf5 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:54.207770 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5e865a-f33c-45c7-9025-411edb9bfbf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:54.208078 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5e865a-f33c-45c7-9025-411edb9bfbf5 None admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:54.208838 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5e865a-f33c-45c7-9025-411edb9bfbf5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:07:54.209129 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5e865a-f33c-45c7-9025-411edb9bfbf5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:54.210502 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5e865a-f33c-45c7-9025-411edb9bfbf5 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:54.219208 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 555/1102] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Jun 29 11:07:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:07:54.225208 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d537c3-866c-4fbf-afe7-90553779eaaa None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:54.334542 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d537c3-866c-4fbf-afe7-90553779eaaa None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:54.368368 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1fd126-7292-4590-bf5c-28964283a233 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:54.369248 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1fd126-7292-4590-bf5c-28964283a233 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:54.369529 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1fd126-7292-4590-bf5c-28964283a233 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:54.369801 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1fd126-7292-4590-bf5c-28964283a233 None admin] PATH_INFO: `/v3/users/d90e84e759f44197b69e8d41e7dbdff5` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:54.370974 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1fd126-7292-4590-bf5c-28964283a233 None admin] RBAC: Authorizing `identity:get_user(user_id=d90e84e759f44197b69e8d41e7dbdff5)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:54.372554 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1fd126-7292-4590-bf5c-28964283a233 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:54.374218 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 548/1103] 10.222.0.44 () {56 vars in 1177 bytes} [Wed Jun 29 11:07:54 2022] GET /identity/v3/users/d90e84e759f44197b69e8d41e7dbdff5 => generated 285 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 11:07:54.380561 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5e865a-f33c-45c7-9025-411edb9bfbf5 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:54.428826 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:54.476358 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5e865a-f33c-45c7-9025-411edb9bfbf5 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:54.508702 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca32656-7f46-45ad-bc4d-e4b576548e98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:54.509332 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca32656-7f46-45ad-bc4d-e4b576548e98 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:54.509621 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca32656-7f46-45ad-bc4d-e4b576548e98 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:54.509895 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca32656-7f46-45ad-bc4d-e4b576548e98 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:54.585843 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca32656-7f46-45ad-bc4d-e4b576548e98 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:54.587217 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca32656-7f46-45ad-bc4d-e4b576548e98 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:54.637647 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:54.685915 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 556/1104] 10.222.0.44 () {58 vars in 1282 bytes} [Wed Jun 29 11:07:54 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 29 11:07:54.692299 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1fd126-7292-4590-bf5c-28964283a233 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:54.723151 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:54.788816 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1fd126-7292-4590-bf5c-28964283a233 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:54.821486 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84dca70-ea33-4ea6-850f-7c5c248a96ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:54.822327 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84dca70-ea33-4ea6-850f-7c5c248a96ca None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:54.822611 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84dca70-ea33-4ea6-850f-7c5c248a96ca None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:54.822887 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84dca70-ea33-4ea6-850f-7c5c248a96ca None admin] PATH_INFO: `/v3/projects/31fa73283b1a4cffae7754d453ae4689` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:54.824077 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84dca70-ea33-4ea6-850f-7c5c248a96ca None admin] RBAC: Authorizing `identity:get_project(project_id=31fa73283b1a4cffae7754d453ae4689)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:54.825488 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84dca70-ea33-4ea6-850f-7c5c248a96ca None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:54.827162 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 549/1105] 10.222.0.44 () {56 vars in 1186 bytes} [Wed Jun 29 11:07:54 2022] GET /identity/v3/projects/31fa73283b1a4cffae7754d453ae4689 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:07:54.833542 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca32656-7f46-45ad-bc4d-e4b576548e98 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:54.931479 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca32656-7f46-45ad-bc4d-e4b576548e98 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:54.965220 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4208a2cc-c372-4eac-9de0-4b1590df73ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:54.965864 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4208a2cc-c372-4eac-9de0-4b1590df73ff None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:54.966149 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4208a2cc-c372-4eac-9de0-4b1590df73ff None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:54.966425 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4208a2cc-c372-4eac-9de0-4b1590df73ff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:54.967247 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4208a2cc-c372-4eac-9de0-4b1590df73ff None admin] RBAC: Adding query filter params (role.id=e95c2ce63e114cc19a9dda6ad21e794f, scope.project.id=31fa73283b1a4cffae7754d453ae4689, user.id=d90e84e759f44197b69e8d41e7dbdff5) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:07:54.967557 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4208a2cc-c372-4eac-9de0-4b1590df73ff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:54.969012 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4208a2cc-c372-4eac-9de0-4b1590df73ff None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:54.974681 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 557/1106] 10.222.0.44 () {56 vars in 1374 bytes} [Wed Jun 29 11:07:54 2022] GET /identity/v3/role_assignments?user.id=d90e84e759f44197b69e8d41e7dbdff5&scope.project.id=31fa73283b1a4cffae7754d453ae4689&role.id=e95c2ce63e114cc19a9dda6ad21e794f => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:07:57.017961 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2362dd-803b-40f1-ae4f-881cb3f6fe75 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:57.018480 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2362dd-803b-40f1-ae4f-881cb3f6fe75 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:57.018572 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2362dd-803b-40f1-ae4f-881cb3f6fe75 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:57.019889 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 550/1107] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:07:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:07:57.025003 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47769c11-b892-4a42-9195-2b021bf31d49 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:57.025365 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47769c11-b892-4a42-9195-2b021bf31d49 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:57.025563 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47769c11-b892-4a42-9195-2b021bf31d49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:57.042969 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-47769c11-b892-4a42-9195-2b021bf31d49 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:07:57.044177 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-47769c11-b892-4a42-9195-2b021bf31d49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:07:57.053580 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:57.107378 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 558/1108] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:07:57 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 11:07:57.132021 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2362dd-803b-40f1-ae4f-881cb3f6fe75 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:57.225961 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:57.229624 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2362dd-803b-40f1-ae4f-881cb3f6fe75 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:57.261934 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0124019c-aba8-48b0-8534-272b07076890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:57.262700 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0124019c-aba8-48b0-8534-272b07076890 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:57.262981 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0124019c-aba8-48b0-8534-272b07076890 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:57.263255 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0124019c-aba8-48b0-8534-272b07076890 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:57.264425 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0124019c-aba8-48b0-8534-272b07076890 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:57.265844 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0124019c-aba8-48b0-8534-272b07076890 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:57.267410 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 551/1109] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jun 29 11:07:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:07:57.273093 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47769c11-b892-4a42-9195-2b021bf31d49 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:57.368037 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47769c11-b892-4a42-9195-2b021bf31d49 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:57.400734 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ee85bd-320f-4b9e-8106-5a40eee715d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:57.401396 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ee85bd-320f-4b9e-8106-5a40eee715d4 None admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:57.401677 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ee85bd-320f-4b9e-8106-5a40eee715d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:57.401950 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ee85bd-320f-4b9e-8106-5a40eee715d4 None admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:57.402644 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ee85bd-320f-4b9e-8106-5a40eee715d4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:57.404089 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ee85bd-320f-4b9e-8106-5a40eee715d4 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:57.491406 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:57.496090 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 559/1110] 10.222.0.44 () {60 vars in 1132 bytes} [Wed Jun 29 11:07:57 2022] POST /identity/v3/users => generated 286 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:07:59.544968 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6610838-adf8-4514-b863-781e82ffa164 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:59.545495 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6610838-adf8-4514-b863-781e82ffa164 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:59.545495 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6610838-adf8-4514-b863-781e82ffa164 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:59.546873 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 552/1111] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:07:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:07:59.552405 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489b6d71-c5e0-42f0-a0a2-79c6718605f5 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:59.552837 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489b6d71-c5e0-42f0-a0a2-79c6718605f5 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:59.552966 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489b6d71-c5e0-42f0-a0a2-79c6718605f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:59.569871 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-489b6d71-c5e0-42f0-a0a2-79c6718605f5 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:07:59.571050 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-489b6d71-c5e0-42f0-a0a2-79c6718605f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:07:59.634711 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 560/1112] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:07:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 11:07:59.657621 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6610838-adf8-4514-b863-781e82ffa164 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:59.756085 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6610838-adf8-4514-b863-781e82ffa164 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:59.789642 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a14b6cd-52f3-477c-a8d3-175607936de3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:59.790476 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a14b6cd-52f3-477c-a8d3-175607936de3 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:59.790756 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a14b6cd-52f3-477c-a8d3-175607936de3 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:59.791029 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a14b6cd-52f3-477c-a8d3-175607936de3 None admin] PATH_INFO: `/v3/roles/fc91ce4ad4a44cc3b05d0936754e87ec` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:59.792492 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a14b6cd-52f3-477c-a8d3-175607936de3 None admin] RBAC: Authorizing `identity:get_role(role_id=fc91ce4ad4a44cc3b05d0936754e87ec)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:59.793846 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a14b6cd-52f3-477c-a8d3-175607936de3 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:59.795254 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 553/1113] 10.222.0.44 () {56 vars in 1177 bytes} [Wed Jun 29 11:07:59 2022] GET /identity/v3/roles/fc91ce4ad4a44cc3b05d0936754e87ec => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 11:07:59.801410 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489b6d71-c5e0-42f0-a0a2-79c6718605f5 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:59.897321 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489b6d71-c5e0-42f0-a0a2-79c6718605f5 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:07:59.925630 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:07:59.931028 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6516b2f-1140-482e-87ff-280f2d1fe75e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:07:59.952141 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6516b2f-1140-482e-87ff-280f2d1fe75e None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:07:59.952141 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6516b2f-1140-482e-87ff-280f2d1fe75e None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:07:59.952141 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6516b2f-1140-482e-87ff-280f2d1fe75e None admin] PATH_INFO: `/v3/users/f322422a08914bff883a43389633744e` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:07:59.952141 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6516b2f-1140-482e-87ff-280f2d1fe75e None admin] RBAC: Authorizing `identity:get_user(user_id=f322422a08914bff883a43389633744e)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:07:59.952141 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6516b2f-1140-482e-87ff-280f2d1fe75e None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:07:59.952141 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 561/1114] 10.222.0.44 () {56 vars in 1177 bytes} [Wed Jun 29 11:07:59 2022] GET /identity/v3/users/f322422a08914bff883a43389633744e => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:07:59.954008 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a14b6cd-52f3-477c-a8d3-175607936de3 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:07:59.988116 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:08:00.047890 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a14b6cd-52f3-477c-a8d3-175607936de3 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:08:00.080228 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d32344-592f-421b-95b8-f18421310270 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:08:00.080857 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d32344-592f-421b-95b8-f18421310270 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:08:00.081137 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d32344-592f-421b-95b8-f18421310270 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:08:00.081410 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d32344-592f-421b-95b8-f18421310270 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:08:00.144516 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d32344-592f-421b-95b8-f18421310270 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:08:00.145931 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d32344-592f-421b-95b8-f18421310270 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:08:00.177885 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:08:00.239023 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 554/1115] 10.222.0.44 () {58 vars in 1282 bytes} [Wed Jun 29 11:07:59 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 285 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 29 11:08:00.244758 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6516b2f-1140-482e-87ff-280f2d1fe75e None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:08:00.339095 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6516b2f-1140-482e-87ff-280f2d1fe75e None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:08:00.371198 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d84c87-0253-4d36-a267-03495f0e4f21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:08:00.372062 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d84c87-0253-4d36-a267-03495f0e4f21 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:08:00.372344 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d84c87-0253-4d36-a267-03495f0e4f21 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:08:00.372617 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d84c87-0253-4d36-a267-03495f0e4f21 None admin] PATH_INFO: `/v3/projects/31fa73283b1a4cffae7754d453ae4689` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:08:00.373771 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d84c87-0253-4d36-a267-03495f0e4f21 None admin] RBAC: Authorizing `identity:get_project(project_id=31fa73283b1a4cffae7754d453ae4689)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:08:00.375184 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d84c87-0253-4d36-a267-03495f0e4f21 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:08:00.376901 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 562/1116] 10.222.0.44 () {56 vars in 1186 bytes} [Wed Jun 29 11:08:00 2022] GET /identity/v3/projects/31fa73283b1a4cffae7754d453ae4689 => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:08:00.382749 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d32344-592f-421b-95b8-f18421310270 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:08:00.479113 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d32344-592f-421b-95b8-f18421310270 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:08:00.510859 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d83983c-ad7a-451d-a497-048d5bac6a25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:08:00.511502 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d83983c-ad7a-451d-a497-048d5bac6a25 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:08:00.511785 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d83983c-ad7a-451d-a497-048d5bac6a25 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:08:00.512098 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d83983c-ad7a-451d-a497-048d5bac6a25 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:08:00.512915 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d83983c-ad7a-451d-a497-048d5bac6a25 None admin] RBAC: Adding query filter params (role.id=fc91ce4ad4a44cc3b05d0936754e87ec, scope.project.id=31fa73283b1a4cffae7754d453ae4689, user.id=f322422a08914bff883a43389633744e) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:08:00.513231 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d83983c-ad7a-451d-a497-048d5bac6a25 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:08:00.514644 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d83983c-ad7a-451d-a497-048d5bac6a25 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:08:00.517886 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:08:00.520814 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 555/1117] 10.222.0.44 () {56 vars in 1374 bytes} [Wed Jun 29 11:08:00 2022] GET /identity/v3/role_assignments?user.id=f322422a08914bff883a43389633744e&scope.project.id=31fa73283b1a4cffae7754d453ae4689&role.id=fc91ce4ad4a44cc3b05d0936754e87ec => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:08:02.496578 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2781472-d439-4576-a666-faf567000531 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:08:02.497220 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2781472-d439-4576-a666-faf567000531 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:08:02.497220 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2781472-d439-4576-a666-faf567000531 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:08:02.498475 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 563/1118] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:08:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:08:02.503688 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb26f459-6acc-4700-a4da-5457fedef79b None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:08:02.504114 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb26f459-6acc-4700-a4da-5457fedef79b None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:08:02.504270 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb26f459-6acc-4700-a4da-5457fedef79b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:08:02.521060 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-fb26f459-6acc-4700-a4da-5457fedef79b None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:08:02.522247 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-fb26f459-6acc-4700-a4da-5457fedef79b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:08:02.586583 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 556/1119] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:08:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 11:08:02.609136 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2781472-d439-4576-a666-faf567000531 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:08:02.668474 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:08:02.710527 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2781472-d439-4576-a666-faf567000531 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:08:02.743150 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a311ba54-2fed-4dc0-af24-ae63efce4a83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:08:02.744017 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a311ba54-2fed-4dc0-af24-ae63efce4a83 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:08:02.744300 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a311ba54-2fed-4dc0-af24-ae63efce4a83 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:08:02.744572 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a311ba54-2fed-4dc0-af24-ae63efce4a83 None admin] PATH_INFO: `/v3/roles/fc91ce4ad4a44cc3b05d0936754e87ec` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:08:02.746019 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a311ba54-2fed-4dc0-af24-ae63efce4a83 None admin] RBAC: Authorizing `identity:get_role(role_id=fc91ce4ad4a44cc3b05d0936754e87ec)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:08:02.747411 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a311ba54-2fed-4dc0-af24-ae63efce4a83 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:08:02.748723 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 564/1120] 10.222.0.44 () {56 vars in 1177 bytes} [Wed Jun 29 11:08:02 2022] GET /identity/v3/roles/fc91ce4ad4a44cc3b05d0936754e87ec => generated 223 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:08:02.754503 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb26f459-6acc-4700-a4da-5457fedef79b None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:08:02.850254 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb26f459-6acc-4700-a4da-5457fedef79b None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:08:02.878180 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:08:02.883658 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488626e3-94b7-425d-970e-d1b050679d8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:08:02.884574 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488626e3-94b7-425d-970e-d1b050679d8b None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:08:02.884840 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488626e3-94b7-425d-970e-d1b050679d8b None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:08:02.885109 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488626e3-94b7-425d-970e-d1b050679d8b None admin] PATH_INFO: `/v3/users/f322422a08914bff883a43389633744e` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:08:02.886281 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488626e3-94b7-425d-970e-d1b050679d8b None admin] RBAC: Authorizing `identity:get_user(user_id=f322422a08914bff883a43389633744e)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:08:02.887701 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488626e3-94b7-425d-970e-d1b050679d8b None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:08:02.889330 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 557/1121] 10.222.0.44 () {56 vars in 1177 bytes} [Wed Jun 29 11:08:02 2022] GET /identity/v3/users/f322422a08914bff883a43389633744e => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:08:02.899893 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a311ba54-2fed-4dc0-af24-ae63efce4a83 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:08:02.992892 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a311ba54-2fed-4dc0-af24-ae63efce4a83 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:08:03.015930 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:08:03.026802 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b542165f-c5f1-4fa3-a71e-db724521af8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:08:03.027654 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b542165f-c5f1-4fa3-a71e-db724521af8d None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:08:03.027971 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b542165f-c5f1-4fa3-a71e-db724521af8d None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:08:03.031402 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b542165f-c5f1-4fa3-a71e-db724521af8d None admin] PATH_INFO: `/v3/projects/31fa73283b1a4cffae7754d453ae4689` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:08:03.031402 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b542165f-c5f1-4fa3-a71e-db724521af8d None admin] RBAC: Authorizing `identity:get_project(project_id=31fa73283b1a4cffae7754d453ae4689)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:08:03.031402 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b542165f-c5f1-4fa3-a71e-db724521af8d None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:08:03.032518 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 565/1122] 10.222.0.44 () {56 vars in 1186 bytes} [Wed Jun 29 11:08:02 2022] GET /identity/v3/projects/31fa73283b1a4cffae7754d453ae4689 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:08:03.038564 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488626e3-94b7-425d-970e-d1b050679d8b None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:08:03.134312 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488626e3-94b7-425d-970e-d1b050679d8b None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:08:03.167553 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f13c7b-5b8f-4a53-a600-3df9c1ef9615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:08:03.168463 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f13c7b-5b8f-4a53-a600-3df9c1ef9615 None admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:08:03.168749 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f13c7b-5b8f-4a53-a600-3df9c1ef9615 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:08:03.169024 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f13c7b-5b8f-4a53-a600-3df9c1ef9615 None admin] PATH_INFO: `/v3/projects/31fa73283b1a4cffae7754d453ae4689/users/f322422a08914bff883a43389633744e/roles/fc91ce4ad4a44cc3b05d0936754e87ec` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:08:03.171202 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f13c7b-5b8f-4a53-a600-3df9c1ef9615 None admin] RBAC: Authorizing `identity:create_grant(project_id=31fa73283b1a4cffae7754d453ae4689, user_id=f322422a08914bff883a43389633744e, role_id=fc91ce4ad4a44cc3b05d0936754e87ec)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:08:03.172645 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f13c7b-5b8f-4a53-a600-3df9c1ef9615 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:08:03.238636 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 558/1123] 10.222.0.44 () {58 vars in 1439 bytes} [Wed Jun 29 11:08:03 2022] PUT /identity/v3/projects/31fa73283b1a4cffae7754d453ae4689/users/f322422a08914bff883a43389633744e/roles/fc91ce4ad4a44cc3b05d0936754e87ec => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 11:08:05.372130 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b6c077-d270-4755-b9d5-1d2b3aa9e198 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:08:05.372665 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b6c077-d270-4755-b9d5-1d2b3aa9e198 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:08:05.372763 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b6c077-d270-4755-b9d5-1d2b3aa9e198 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:08:05.374045 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 566/1124] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:08:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:08:05.379748 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c279a6d-0a85-4336-a6c4-f9c0f6765591 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:08:05.380171 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c279a6d-0a85-4336-a6c4-f9c0f6765591 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:08:05.380328 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c279a6d-0a85-4336-a6c4-f9c0f6765591 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:08:05.397351 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-2c279a6d-0a85-4336-a6c4-f9c0f6765591 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:08:05.398546 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-2c279a6d-0a85-4336-a6c4-f9c0f6765591 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:08:05.450954 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:08:05.464629 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 559/1125] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:08:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 11:08:05.489195 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b6c077-d270-4755-b9d5-1d2b3aa9e198 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:08:05.586588 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b6c077-d270-4755-b9d5-1d2b3aa9e198 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:08:05.620873 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f374ba-7cba-46ec-9a24-46e478c21a80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:08:05.621727 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f374ba-7cba-46ec-9a24-46e478c21a80 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:08:05.622013 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f374ba-7cba-46ec-9a24-46e478c21a80 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:08:05.622341 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f374ba-7cba-46ec-9a24-46e478c21a80 None admin] PATH_INFO: `/v3/roles/fc91ce4ad4a44cc3b05d0936754e87ec` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:08:05.623765 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f374ba-7cba-46ec-9a24-46e478c21a80 None admin] RBAC: Authorizing `identity:get_role(role_id=fc91ce4ad4a44cc3b05d0936754e87ec)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:08:05.625158 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f374ba-7cba-46ec-9a24-46e478c21a80 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:08:05.626547 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 567/1126] 10.222.0.44 () {56 vars in 1177 bytes} [Wed Jun 29 11:08:05 2022] GET /identity/v3/roles/fc91ce4ad4a44cc3b05d0936754e87ec => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:08:05.632950 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c279a6d-0a85-4336-a6c4-f9c0f6765591 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:08:05.727844 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c279a6d-0a85-4336-a6c4-f9c0f6765591 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:08:05.760863 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0f4148-0839-4253-acdc-ec6c821d1cdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:08:05.761717 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0f4148-0839-4253-acdc-ec6c821d1cdb None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:08:05.761996 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0f4148-0839-4253-acdc-ec6c821d1cdb None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:08:05.762268 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0f4148-0839-4253-acdc-ec6c821d1cdb None admin] PATH_INFO: `/v3/users/f322422a08914bff883a43389633744e` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:08:05.763437 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0f4148-0839-4253-acdc-ec6c821d1cdb None admin] RBAC: Authorizing `identity:get_user(user_id=f322422a08914bff883a43389633744e)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:08:05.764897 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0f4148-0839-4253-acdc-ec6c821d1cdb None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:08:05.766492 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 560/1127] 10.222.0.44 () {56 vars in 1177 bytes} [Wed Jun 29 11:08:05 2022] GET /identity/v3/users/f322422a08914bff883a43389633744e => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:08:05.772413 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f374ba-7cba-46ec-9a24-46e478c21a80 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:08:05.829829 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:08:05.868445 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f374ba-7cba-46ec-9a24-46e478c21a80 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:08:05.900387 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6912eb-6fb8-41f9-a457-7ffdee55d076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:08:05.900960 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6912eb-6fb8-41f9-a457-7ffdee55d076 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:08:05.901240 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6912eb-6fb8-41f9-a457-7ffdee55d076 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:08:05.901514 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6912eb-6fb8-41f9-a457-7ffdee55d076 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:08:05.966177 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6912eb-6fb8-41f9-a457-7ffdee55d076 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:08:05.967621 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6912eb-6fb8-41f9-a457-7ffdee55d076 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:08:06.021769 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:08:06.061690 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 568/1128] 10.222.0.44 () {58 vars in 1282 bytes} [Wed Jun 29 11:08:05 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 29 11:08:06.067563 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0f4148-0839-4253-acdc-ec6c821d1cdb None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:08:06.121793 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:08:06.162710 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0f4148-0839-4253-acdc-ec6c821d1cdb None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:08:06.195500 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f5cdef-51ef-48c9-a84f-9b40744b320b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:08:06.196627 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f5cdef-51ef-48c9-a84f-9b40744b320b None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:08:06.196908 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f5cdef-51ef-48c9-a84f-9b40744b320b None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:08:06.197185 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f5cdef-51ef-48c9-a84f-9b40744b320b None admin] PATH_INFO: `/v3/projects/31fa73283b1a4cffae7754d453ae4689` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:08:06.198346 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f5cdef-51ef-48c9-a84f-9b40744b320b None admin] RBAC: Authorizing `identity:get_project(project_id=31fa73283b1a4cffae7754d453ae4689)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:08:06.199754 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f5cdef-51ef-48c9-a84f-9b40744b320b None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:08:06.201443 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 561/1129] 10.222.0.44 () {56 vars in 1186 bytes} [Wed Jun 29 11:08:06 2022] GET /identity/v3/projects/31fa73283b1a4cffae7754d453ae4689 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:08:06.208136 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6912eb-6fb8-41f9-a457-7ffdee55d076 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:08:06.302289 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6912eb-6fb8-41f9-a457-7ffdee55d076 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:08:06.334889 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d734af43-09c5-4987-beb5-e86f13405236 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:08:06.335525 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d734af43-09c5-4987-beb5-e86f13405236 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:08:06.335807 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d734af43-09c5-4987-beb5-e86f13405236 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:08:06.336110 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d734af43-09c5-4987-beb5-e86f13405236 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:08:06.336928 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d734af43-09c5-4987-beb5-e86f13405236 None admin] RBAC: Adding query filter params (role.id=fc91ce4ad4a44cc3b05d0936754e87ec, scope.project.id=31fa73283b1a4cffae7754d453ae4689, user.id=f322422a08914bff883a43389633744e) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:08:06.337243 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d734af43-09c5-4987-beb5-e86f13405236 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:08:06.338662 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d734af43-09c5-4987-beb5-e86f13405236 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:08:06.344301 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 569/1130] 10.222.0.44 () {56 vars in 1374 bytes} [Wed Jun 29 11:08:06 2022] GET /identity/v3/role_assignments?user.id=f322422a08914bff883a43389633744e&scope.project.id=31fa73283b1a4cffae7754d453ae4689&role.id=fc91ce4ad4a44cc3b05d0936754e87ec => generated 608 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 11:08:08.361580 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a3e3a7-913d-462a-856f-6e678bb999e1 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:08:08.362105 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a3e3a7-913d-462a-856f-6e678bb999e1 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:08:08.362105 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a3e3a7-913d-462a-856f-6e678bb999e1 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:08:08.363504 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 562/1131] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:08:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:08:08.368694 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5ce5af-ec51-4b68-aafb-8000a34bdfec None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:08:08.369104 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5ce5af-ec51-4b68-aafb-8000a34bdfec None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:08:08.369261 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5ce5af-ec51-4b68-aafb-8000a34bdfec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:08:08.386882 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-7a5ce5af-ec51-4b68-aafb-8000a34bdfec None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:08:08.388140 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-7a5ce5af-ec51-4b68-aafb-8000a34bdfec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:08:08.405511 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:08:08.452987 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 570/1132] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:08:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 11:08:08.476565 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a3e3a7-913d-462a-856f-6e678bb999e1 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:08:08.575607 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a3e3a7-913d-462a-856f-6e678bb999e1 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:08:08.597895 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:08:08.608120 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e565a9-6ef7-4914-8f33-cc7a98ef19bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:08:08.608910 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e565a9-6ef7-4914-8f33-cc7a98ef19bb None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:08:08.609196 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e565a9-6ef7-4914-8f33-cc7a98ef19bb None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:08:08.609475 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e565a9-6ef7-4914-8f33-cc7a98ef19bb None admin] PATH_INFO: `/v3/domains/default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:08:08.610646 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e565a9-6ef7-4914-8f33-cc7a98ef19bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:08:08.612702 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e565a9-6ef7-4914-8f33-cc7a98ef19bb None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:08:08.614272 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 563/1133] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jun 29 11:08:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 11:08:08.620006 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5ce5af-ec51-4b68-aafb-8000a34bdfec None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:08:08.720942 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5ce5af-ec51-4b68-aafb-8000a34bdfec None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:08:08.753350 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4290090a-75d0-4423-ac5c-456cd264330e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:08:08.753999 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4290090a-75d0-4423-ac5c-456cd264330e None admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:08:08.754278 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4290090a-75d0-4423-ac5c-456cd264330e None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:08:08.754552 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4290090a-75d0-4423-ac5c-456cd264330e None admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:08:08.755256 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4290090a-75d0-4423-ac5c-456cd264330e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:08:08.756712 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4290090a-75d0-4423-ac5c-456cd264330e None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:08:08.858962 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 571/1134] 10.222.0.44 () {60 vars in 1141 bytes} [Wed Jun 29 11:08:08 2022] POST /identity/v3/projects => generated 311 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:08:10.886231 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a67a30-3afc-494e-9d2e-d535ea5d01ff None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:08:10.886783 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a67a30-3afc-494e-9d2e-d535ea5d01ff None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:08:10.886783 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a67a30-3afc-494e-9d2e-d535ea5d01ff None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:08:10.888184 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 564/1135] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:08:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:08:10.893944 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b5dd7b-219b-44af-baa5-7822bba9b7fc None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:08:10.894309 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b5dd7b-219b-44af-baa5-7822bba9b7fc None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:08:10.894503 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b5dd7b-219b-44af-baa5-7822bba9b7fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:08:10.905756 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:08:10.912025 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-e4b5dd7b-219b-44af-baa5-7822bba9b7fc None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:08:10.913202 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-e4b5dd7b-219b-44af-baa5-7822bba9b7fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:08:10.987962 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 572/1136] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:08:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 11:08:11.014109 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a67a30-3afc-494e-9d2e-d535ea5d01ff None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:08:11.112945 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a67a30-3afc-494e-9d2e-d535ea5d01ff None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:08:11.146195 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009b34f4-90a6-4b5d-9d50-997acc724b4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:08:11.146914 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009b34f4-90a6-4b5d-9d50-997acc724b4a None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:08:11.147195 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009b34f4-90a6-4b5d-9d50-997acc724b4a None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:08:11.147471 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009b34f4-90a6-4b5d-9d50-997acc724b4a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:08:11.148642 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009b34f4-90a6-4b5d-9d50-997acc724b4a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:08:11.150069 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009b34f4-90a6-4b5d-9d50-997acc724b4a None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:08:11.151647 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 565/1137] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jun 29 11:08:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:08:11.157404 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b5dd7b-219b-44af-baa5-7822bba9b7fc None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:08:11.253335 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b5dd7b-219b-44af-baa5-7822bba9b7fc None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:08:11.286322 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec3da64-5714-4088-9ba3-e69a02930803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:08:11.286973 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec3da64-5714-4088-9ba3-e69a02930803 None admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:08:11.287258 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec3da64-5714-4088-9ba3-e69a02930803 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:08:11.287531 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec3da64-5714-4088-9ba3-e69a02930803 None admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:08:11.288261 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec3da64-5714-4088-9ba3-e69a02930803 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:08:11.289686 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec3da64-5714-4088-9ba3-e69a02930803 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:08:11.296894 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:08:11.401549 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 573/1138] 10.222.0.44 () {60 vars in 1132 bytes} [Wed Jun 29 11:08:11 2022] POST /identity/v3/users => generated 286 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:08:13.455130 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eee4df-3c6f-4338-9392-99c67f74a960 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:08:13.455652 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eee4df-3c6f-4338-9392-99c67f74a960 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:08:13.455652 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eee4df-3c6f-4338-9392-99c67f74a960 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:08:13.457082 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 566/1139] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:08:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:08:13.462280 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef13139-eed6-433a-b9bf-73b1c31d5282 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:08:13.462694 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef13139-eed6-433a-b9bf-73b1c31d5282 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:08:13.462841 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef13139-eed6-433a-b9bf-73b1c31d5282 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:08:13.479070 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-5ef13139-eed6-433a-b9bf-73b1c31d5282 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:08:13.480286 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-5ef13139-eed6-433a-b9bf-73b1c31d5282 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:08:13.545082 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 574/1140] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:08:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 11:08:13.569367 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65eee4df-3c6f-4338-9392-99c67f74a960 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:08:13.628109 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:08:13.666696 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65eee4df-3c6f-4338-9392-99c67f74a960 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:08:13.699207 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a2c6b6-fbd8-467b-9e7c-ca6ce08a8057 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:08:13.700013 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a2c6b6-fbd8-467b-9e7c-ca6ce08a8057 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:08:13.700297 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a2c6b6-fbd8-467b-9e7c-ca6ce08a8057 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:08:13.700568 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a2c6b6-fbd8-467b-9e7c-ca6ce08a8057 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:08:13.710880 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a2c6b6-fbd8-467b-9e7c-ca6ce08a8057 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 11:08:13.718470 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-07a2c6b6-fbd8-467b-9e7c-ca6ce08a8057 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 11:08:13.718470 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:08:13.718470 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:08:13.718470 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:08:13.718470 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:08:13.718470 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:08:13.718470 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:08:13.718470 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:08:13.718470 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:08:13.718470 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:08:13.718470 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:08:13.718470 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:08:13.718470 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 11:08:13.718470 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 11:08:13.718470 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 11:08:13.718470 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 11:08:13.719610 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 11:08:13.719610 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 11:08:13.719610 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 11:08:13.719610 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 11:08:13.719610 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:08:13.719610 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:08:13.719610 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:08:13.719610 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:08:13.719610 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:08:13.719610 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:08:13.719610 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:08:13.719610 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jun 29 11:08:13.719610 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:08:13.719610 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:08:13.719610 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:08:13.719610 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:08:13.721369 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:08:13.721369 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:08:13.721369 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:08:13.721369 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:08:13.721369 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 11:08:13.721369 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 11:08:13.721369 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 11:08:13.721369 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 11:08:13.721369 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 11:08:13.721369 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 11:08:13.721369 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 11:08:13.721369 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:08:13.721369 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 567/1141] 10.222.0.44 () {56 vars in 1096 bytes} [Wed Jun 29 11:08:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:08:13.726130 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef13139-eed6-433a-b9bf-73b1c31d5282 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:08:13.819594 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef13139-eed6-433a-b9bf-73b1c31d5282 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:08:13.841369 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:08:13.852960 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ee8192-1abc-42f3-b054-7e852680f35e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:08:13.853616 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ee8192-1abc-42f3-b054-7e852680f35e None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:08:13.853901 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ee8192-1abc-42f3-b054-7e852680f35e None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:08:13.854180 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ee8192-1abc-42f3-b054-7e852680f35e None admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:08:13.854926 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ee8192-1abc-42f3-b054-7e852680f35e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:08:13.855233 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ee8192-1abc-42f3-b054-7e852680f35e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:08:13.856665 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ee8192-1abc-42f3-b054-7e852680f35e None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:08:13.864964 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 575/1142] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Jun 29 11:08:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:08:13.870539 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a2c6b6-fbd8-467b-9e7c-ca6ce08a8057 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:08:13.969189 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a2c6b6-fbd8-467b-9e7c-ca6ce08a8057 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:08:13.988700 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:08:14.001796 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bf4970-456e-4e76-8948-3b015f3ba030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:08:14.002654 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bf4970-456e-4e76-8948-3b015f3ba030 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:08:14.002935 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bf4970-456e-4e76-8948-3b015f3ba030 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:08:14.003210 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bf4970-456e-4e76-8948-3b015f3ba030 None admin] PATH_INFO: `/v3/users/544bef3a9df841a8a3d3351c15d0044a` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:08:14.015791 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bf4970-456e-4e76-8948-3b015f3ba030 None admin] RBAC: Authorizing `identity:get_user(user_id=544bef3a9df841a8a3d3351c15d0044a)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:08:14.017565 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bf4970-456e-4e76-8948-3b015f3ba030 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:08:14.019191 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 568/1143] 10.222.0.44 () {56 vars in 1177 bytes} [Wed Jun 29 11:08:13 2022] GET /identity/v3/users/544bef3a9df841a8a3d3351c15d0044a => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:08:14.024979 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ee8192-1abc-42f3-b054-7e852680f35e None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:08:14.121163 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ee8192-1abc-42f3-b054-7e852680f35e None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:08:14.152896 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353b7ae7-0dd7-49a9-a3cf-adfed0bf9813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:08:14.153533 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353b7ae7-0dd7-49a9-a3cf-adfed0bf9813 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:08:14.153818 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353b7ae7-0dd7-49a9-a3cf-adfed0bf9813 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:08:14.154093 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353b7ae7-0dd7-49a9-a3cf-adfed0bf9813 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:08:14.202955 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:08:14.217919 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353b7ae7-0dd7-49a9-a3cf-adfed0bf9813 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:08:14.224282 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353b7ae7-0dd7-49a9-a3cf-adfed0bf9813 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:08:14.312295 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 576/1144] 10.222.0.44 () {58 vars in 1282 bytes} [Wed Jun 29 11:08:14 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 29 11:08:14.318126 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bf4970-456e-4e76-8948-3b015f3ba030 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:08:14.416642 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bf4970-456e-4e76-8948-3b015f3ba030 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:08:14.448868 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd6a495-35f3-4108-b484-e4e8de097ad4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:08:14.449704 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd6a495-35f3-4108-b484-e4e8de097ad4 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:08:14.450005 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd6a495-35f3-4108-b484-e4e8de097ad4 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:08:14.450275 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd6a495-35f3-4108-b484-e4e8de097ad4 None admin] PATH_INFO: `/v3/projects/986d93889c074f0b81463da0ecff41fb` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:08:14.451423 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd6a495-35f3-4108-b484-e4e8de097ad4 None admin] RBAC: Authorizing `identity:get_project(project_id=986d93889c074f0b81463da0ecff41fb)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:08:14.452862 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd6a495-35f3-4108-b484-e4e8de097ad4 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:08:14.454535 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 569/1145] 10.222.0.44 () {56 vars in 1186 bytes} [Wed Jun 29 11:08:14 2022] GET /identity/v3/projects/986d93889c074f0b81463da0ecff41fb => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:08:14.460981 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353b7ae7-0dd7-49a9-a3cf-adfed0bf9813 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:08:14.539772 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:08:14.554353 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353b7ae7-0dd7-49a9-a3cf-adfed0bf9813 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:08:14.587511 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098ab0fa-bb2b-47c5-a471-c1cc6f6d07aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:08:14.588176 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098ab0fa-bb2b-47c5-a471-c1cc6f6d07aa None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:08:14.588456 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098ab0fa-bb2b-47c5-a471-c1cc6f6d07aa None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:08:14.588728 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098ab0fa-bb2b-47c5-a471-c1cc6f6d07aa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:08:14.589543 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098ab0fa-bb2b-47c5-a471-c1cc6f6d07aa None admin] RBAC: Adding query filter params (role.id=e95c2ce63e114cc19a9dda6ad21e794f, scope.project.id=986d93889c074f0b81463da0ecff41fb, user.id=544bef3a9df841a8a3d3351c15d0044a) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:08:14.589851 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098ab0fa-bb2b-47c5-a471-c1cc6f6d07aa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:08:14.591261 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098ab0fa-bb2b-47c5-a471-c1cc6f6d07aa None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:08:14.596514 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 577/1146] 10.222.0.44 () {56 vars in 1374 bytes} [Wed Jun 29 11:08:14 2022] GET /identity/v3/role_assignments?user.id=544bef3a9df841a8a3d3351c15d0044a&scope.project.id=986d93889c074f0b81463da0ecff41fb&role.id=e95c2ce63e114cc19a9dda6ad21e794f => generated 259 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:08:16.569146 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b93ebb-d21f-46cd-a710-60373fcf62e1 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:08:16.569673 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b93ebb-d21f-46cd-a710-60373fcf62e1 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:08:16.569673 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b93ebb-d21f-46cd-a710-60373fcf62e1 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:08:16.571293 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 570/1147] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:08:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:08:16.576415 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d66673-b3eb-493f-934d-4985fcc9e83b None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:08:16.576801 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d66673-b3eb-493f-934d-4985fcc9e83b None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:08:16.576971 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d66673-b3eb-493f-934d-4985fcc9e83b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:08:16.593740 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-24d66673-b3eb-493f-934d-4985fcc9e83b None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:08:16.594916 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-24d66673-b3eb-493f-934d-4985fcc9e83b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:08:16.657852 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 578/1148] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:08:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 11:08:16.680210 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b93ebb-d21f-46cd-a710-60373fcf62e1 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:08:16.735711 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:08:16.787809 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b93ebb-d21f-46cd-a710-60373fcf62e1 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:08:16.820443 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380e8581-9fd8-4c00-9e34-182c0245bc1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:08:16.821431 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380e8581-9fd8-4c00-9e34-182c0245bc1f None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:08:16.821715 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380e8581-9fd8-4c00-9e34-182c0245bc1f None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:08:16.821991 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380e8581-9fd8-4c00-9e34-182c0245bc1f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:08:16.831791 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380e8581-9fd8-4c00-9e34-182c0245bc1f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 11:08:16.839464 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-380e8581-9fd8-4c00-9e34-182c0245bc1f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 11:08:16.839464 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:08:16.839464 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:08:16.839464 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:08:16.839464 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:08:16.839464 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:08:16.839464 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:08:16.839464 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:08:16.839464 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:08:16.839464 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:08:16.839464 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:08:16.839464 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:08:16.839464 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 11:08:16.839464 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 11:08:16.839464 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 11:08:16.839464 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 11:08:16.840499 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 11:08:16.840499 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 11:08:16.840499 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 11:08:16.840499 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 11:08:16.840499 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:08:16.840499 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:08:16.840499 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:08:16.840499 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:08:16.840499 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:08:16.840499 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:08:16.840499 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:08:16.840499 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jun 29 11:08:16.840499 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:08:16.840499 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:08:16.840499 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:08:16.840499 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:08:16.841402 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:08:16.841402 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:08:16.841402 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:08:16.841402 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:08:16.841402 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 11:08:16.841402 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 11:08:16.841402 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 11:08:16.841402 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 11:08:16.841402 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 11:08:16.841402 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 11:08:16.841402 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 11:08:16.841402 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:08:16.841402 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 571/1149] 10.222.0.44 () {56 vars in 1096 bytes} [Wed Jun 29 11:08:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:08:16.847445 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d66673-b3eb-493f-934d-4985fcc9e83b None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:08:16.922591 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:08:16.943415 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d66673-b3eb-493f-934d-4985fcc9e83b None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:08:16.976679 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de00cf0-936e-4b44-bc0b-3751ce0ba649 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:08:16.977333 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de00cf0-936e-4b44-bc0b-3751ce0ba649 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:08:16.977611 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de00cf0-936e-4b44-bc0b-3751ce0ba649 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:08:16.977882 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de00cf0-936e-4b44-bc0b-3751ce0ba649 None admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:08:16.978617 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de00cf0-936e-4b44-bc0b-3751ce0ba649 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:08:16.978921 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de00cf0-936e-4b44-bc0b-3751ce0ba649 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:08:16.980312 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de00cf0-936e-4b44-bc0b-3751ce0ba649 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:08:16.988639 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 579/1150] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Jun 29 11:08:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:08:16.994511 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380e8581-9fd8-4c00-9e34-182c0245bc1f None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:08:17.090854 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380e8581-9fd8-4c00-9e34-182c0245bc1f None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:08:17.097946 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:08:17.123394 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ab55a0-d137-4778-9d0b-d3378b6f6a72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:08:17.124299 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ab55a0-d137-4778-9d0b-d3378b6f6a72 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:08:17.124582 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ab55a0-d137-4778-9d0b-d3378b6f6a72 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:08:17.124856 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ab55a0-d137-4778-9d0b-d3378b6f6a72 None admin] PATH_INFO: `/v3/users/544bef3a9df841a8a3d3351c15d0044a` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:08:17.126025 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ab55a0-d137-4778-9d0b-d3378b6f6a72 None admin] RBAC: Authorizing `identity:get_user(user_id=544bef3a9df841a8a3d3351c15d0044a)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:08:17.127512 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ab55a0-d137-4778-9d0b-d3378b6f6a72 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:08:17.129131 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 572/1151] 10.222.0.44 () {56 vars in 1177 bytes} [Wed Jun 29 11:08:16 2022] GET /identity/v3/users/544bef3a9df841a8a3d3351c15d0044a => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:08:17.135080 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de00cf0-936e-4b44-bc0b-3751ce0ba649 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:08:17.231325 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de00cf0-936e-4b44-bc0b-3751ce0ba649 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:08:17.262416 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2356a6f-b712-4838-929e-92f3eb4ae878 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:08:17.263242 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2356a6f-b712-4838-929e-92f3eb4ae878 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:08:17.263522 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2356a6f-b712-4838-929e-92f3eb4ae878 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:08:17.263793 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2356a6f-b712-4838-929e-92f3eb4ae878 None admin] PATH_INFO: `/v3/projects/986d93889c074f0b81463da0ecff41fb` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:08:17.264991 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2356a6f-b712-4838-929e-92f3eb4ae878 None admin] RBAC: Authorizing `identity:get_project(project_id=986d93889c074f0b81463da0ecff41fb)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:08:17.266394 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2356a6f-b712-4838-929e-92f3eb4ae878 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:08:17.268585 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 580/1152] 10.222.0.44 () {56 vars in 1186 bytes} [Wed Jun 29 11:08:17 2022] GET /identity/v3/projects/986d93889c074f0b81463da0ecff41fb => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:08:17.274299 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ab55a0-d137-4778-9d0b-d3378b6f6a72 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:08:17.370918 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ab55a0-d137-4778-9d0b-d3378b6f6a72 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:08:17.402874 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd90439-8f83-40ec-8829-21547801a8e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:08:17.403735 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd90439-8f83-40ec-8829-21547801a8e5 None admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:08:17.404056 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd90439-8f83-40ec-8829-21547801a8e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:08:17.404331 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd90439-8f83-40ec-8829-21547801a8e5 None admin] PATH_INFO: `/v3/projects/986d93889c074f0b81463da0ecff41fb/users/544bef3a9df841a8a3d3351c15d0044a/roles/e95c2ce63e114cc19a9dda6ad21e794f` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:08:17.406278 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd90439-8f83-40ec-8829-21547801a8e5 None admin] RBAC: Authorizing `identity:create_grant(project_id=986d93889c074f0b81463da0ecff41fb, user_id=544bef3a9df841a8a3d3351c15d0044a, role_id=e95c2ce63e114cc19a9dda6ad21e794f)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:08:17.407683 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd90439-8f83-40ec-8829-21547801a8e5 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:08:17.497772 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 573/1153] 10.222.0.44 () {58 vars in 1439 bytes} [Wed Jun 29 11:08:17 2022] PUT /identity/v3/projects/986d93889c074f0b81463da0ecff41fb/users/544bef3a9df841a8a3d3351c15d0044a/roles/e95c2ce63e114cc19a9dda6ad21e794f => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:08:19.487269 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915e0117-e6c4-40d2-b08d-0c4f6bf181c5 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:08:19.487797 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915e0117-e6c4-40d2-b08d-0c4f6bf181c5 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:08:19.487797 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915e0117-e6c4-40d2-b08d-0c4f6bf181c5 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:08:19.489201 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 581/1154] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:08:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:08:19.494483 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6528c2fb-57a0-4a96-8019-2045dc0625e2 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:08:19.494859 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6528c2fb-57a0-4a96-8019-2045dc0625e2 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:08:19.495047 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6528c2fb-57a0-4a96-8019-2045dc0625e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:08:19.511318 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-6528c2fb-57a0-4a96-8019-2045dc0625e2 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:08:19.512542 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-6528c2fb-57a0-4a96-8019-2045dc0625e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:08:19.539239 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:08:19.577651 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 574/1155] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:08:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 11:08:19.601844 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915e0117-e6c4-40d2-b08d-0c4f6bf181c5 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:08:19.613650 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:08:19.700802 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915e0117-e6c4-40d2-b08d-0c4f6bf181c5 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:08:19.734213 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00033524-996b-46d6-8257-c1ae95785bbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:08:19.734995 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00033524-996b-46d6-8257-c1ae95785bbd None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:08:19.735278 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00033524-996b-46d6-8257-c1ae95785bbd None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:08:19.735551 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00033524-996b-46d6-8257-c1ae95785bbd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:08:19.745859 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00033524-996b-46d6-8257-c1ae95785bbd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 11:08:19.753905 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-00033524-996b-46d6-8257-c1ae95785bbd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 11:08:19.753905 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:08:19.753905 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:08:19.753905 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:08:19.753905 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:08:19.753905 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:08:19.753905 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:08:19.753905 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:08:19.753905 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:08:19.753905 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:08:19.753905 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:08:19.753905 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:08:19.753905 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 11:08:19.753905 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 11:08:19.753905 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 11:08:19.753905 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 11:08:19.755036 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 11:08:19.755036 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 11:08:19.755036 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 11:08:19.755036 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 11:08:19.755036 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:08:19.755036 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:08:19.755036 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:08:19.755036 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:08:19.755036 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:08:19.755036 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:08:19.755036 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:08:19.755036 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jun 29 11:08:19.755036 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:08:19.755036 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:08:19.755036 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:08:19.755036 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:08:19.756122 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:08:19.756122 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:08:19.756122 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:08:19.756122 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:08:19.756122 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 11:08:19.756122 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 11:08:19.756122 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 11:08:19.756122 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 11:08:19.756122 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 11:08:19.756122 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 11:08:19.756122 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 11:08:19.756122 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jun 29 11:08:19.756122 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 582/1156] 10.222.0.44 () {56 vars in 1096 bytes} [Wed Jun 29 11:08:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:08:19.761144 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6528c2fb-57a0-4a96-8019-2045dc0625e2 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:08:19.856219 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6528c2fb-57a0-4a96-8019-2045dc0625e2 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:08:19.888916 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5839689e-00fe-4748-8acb-28818608e269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:08:19.889568 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5839689e-00fe-4748-8acb-28818608e269 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:08:19.889848 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5839689e-00fe-4748-8acb-28818608e269 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:08:19.890120 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5839689e-00fe-4748-8acb-28818608e269 None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:08:19.890854 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5839689e-00fe-4748-8acb-28818608e269 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:08:19.891161 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5839689e-00fe-4748-8acb-28818608e269 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:08:19.892584 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5839689e-00fe-4748-8acb-28818608e269 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:08:19.900448 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 575/1157] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Jun 29 11:08:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:08:19.906022 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00033524-996b-46d6-8257-c1ae95785bbd None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:08:19.970958 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:08:20.002349 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00033524-996b-46d6-8257-c1ae95785bbd None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:08:20.035462 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d035e6bf-dac7-464a-a38f-f7b8148e1234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:08:20.036348 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d035e6bf-dac7-464a-a38f-f7b8148e1234 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:08:20.036626 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d035e6bf-dac7-464a-a38f-f7b8148e1234 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:08:20.036897 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d035e6bf-dac7-464a-a38f-f7b8148e1234 None admin] PATH_INFO: `/v3/users/544bef3a9df841a8a3d3351c15d0044a` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:08:20.038072 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d035e6bf-dac7-464a-a38f-f7b8148e1234 None admin] RBAC: Authorizing `identity:get_user(user_id=544bef3a9df841a8a3d3351c15d0044a)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:08:20.039496 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d035e6bf-dac7-464a-a38f-f7b8148e1234 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:08:20.041142 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 583/1158] 10.222.0.44 () {56 vars in 1177 bytes} [Wed Jun 29 11:08:19 2022] GET /identity/v3/users/544bef3a9df841a8a3d3351c15d0044a => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:08:20.047336 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5839689e-00fe-4748-8acb-28818608e269 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:08:20.082485 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:08:20.151788 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5839689e-00fe-4748-8acb-28818608e269 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:08:20.184559 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d88ee5f-f881-4e77-b4de-85df43effbf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:08:20.185197 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d88ee5f-f881-4e77-b4de-85df43effbf3 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:08:20.185481 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d88ee5f-f881-4e77-b4de-85df43effbf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:08:20.185757 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d88ee5f-f881-4e77-b4de-85df43effbf3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:08:20.249859 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d88ee5f-f881-4e77-b4de-85df43effbf3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:08:20.251272 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d88ee5f-f881-4e77-b4de-85df43effbf3 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:08:20.276171 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:08:20.346783 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 576/1159] 10.222.0.44 () {58 vars in 1282 bytes} [Wed Jun 29 11:08:20 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 29 11:08:20.354542 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d035e6bf-dac7-464a-a38f-f7b8148e1234 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:08:20.449647 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d035e6bf-dac7-464a-a38f-f7b8148e1234 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:08:20.482591 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6280679-3c64-4feb-846b-d4f7c8e27b2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:08:20.483424 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6280679-3c64-4feb-846b-d4f7c8e27b2a None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:08:20.483705 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6280679-3c64-4feb-846b-d4f7c8e27b2a None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:08:20.484009 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6280679-3c64-4feb-846b-d4f7c8e27b2a None admin] PATH_INFO: `/v3/projects/986d93889c074f0b81463da0ecff41fb` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:08:20.485170 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6280679-3c64-4feb-846b-d4f7c8e27b2a None admin] RBAC: Authorizing `identity:get_project(project_id=986d93889c074f0b81463da0ecff41fb)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:08:20.486637 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6280679-3c64-4feb-846b-d4f7c8e27b2a None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:08:20.488332 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 584/1160] 10.222.0.44 () {56 vars in 1186 bytes} [Wed Jun 29 11:08:20 2022] GET /identity/v3/projects/986d93889c074f0b81463da0ecff41fb => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:08:20.494251 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d88ee5f-f881-4e77-b4de-85df43effbf3 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:08:20.590540 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d88ee5f-f881-4e77-b4de-85df43effbf3 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:08:20.614110 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:08:20.622111 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1ff02d-008e-4f98-8b3b-d4d843173b2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:08:20.631108 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1ff02d-008e-4f98-8b3b-d4d843173b2a None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:08:20.631108 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1ff02d-008e-4f98-8b3b-d4d843173b2a None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:08:20.631108 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1ff02d-008e-4f98-8b3b-d4d843173b2a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:08:20.631108 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1ff02d-008e-4f98-8b3b-d4d843173b2a None admin] RBAC: Adding query filter params (role.id=e95c2ce63e114cc19a9dda6ad21e794f, scope.project.id=986d93889c074f0b81463da0ecff41fb, user.id=544bef3a9df841a8a3d3351c15d0044a) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:08:20.631108 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1ff02d-008e-4f98-8b3b-d4d843173b2a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:08:20.631108 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1ff02d-008e-4f98-8b3b-d4d843173b2a None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:08:20.631911 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 577/1161] 10.222.0.44 () {56 vars in 1374 bytes} [Wed Jun 29 11:08:20 2022] GET /identity/v3/role_assignments?user.id=544bef3a9df841a8a3d3351c15d0044a&scope.project.id=986d93889c074f0b81463da0ecff41fb&role.id=e95c2ce63e114cc19a9dda6ad21e794f => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:08:22.686828 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbdf379-802b-4bb3-9dc9-b1ff63cbfadb None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:08:22.687355 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbdf379-802b-4bb3-9dc9-b1ff63cbfadb None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:08:22.687355 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbdf379-802b-4bb3-9dc9-b1ff63cbfadb None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:08:22.688758 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 585/1162] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:08:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:08:22.693880 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486c2736-66bd-4887-aae7-8687c5c9979e None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:08:22.694305 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486c2736-66bd-4887-aae7-8687c5c9979e None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:08:22.694440 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486c2736-66bd-4887-aae7-8687c5c9979e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:08:22.711633 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-486c2736-66bd-4887-aae7-8687c5c9979e None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:08:22.712842 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-486c2736-66bd-4887-aae7-8687c5c9979e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:08:22.781192 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 578/1163] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:08:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 11:08:22.803975 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbdf379-802b-4bb3-9dc9-b1ff63cbfadb None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:08:22.807711 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:08:22.901881 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbdf379-802b-4bb3-9dc9-b1ff63cbfadb None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:08:22.934142 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5a6f49-6b0b-47cf-87ae-8206cb11b8b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:08:22.934783 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5a6f49-6b0b-47cf-87ae-8206cb11b8b6 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:08:22.935066 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5a6f49-6b0b-47cf-87ae-8206cb11b8b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:08:22.935341 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5a6f49-6b0b-47cf-87ae-8206cb11b8b6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:08:22.999112 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5a6f49-6b0b-47cf-87ae-8206cb11b8b6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:08:23.000514 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5a6f49-6b0b-47cf-87ae-8206cb11b8b6 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:08:23.003500 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:08:23.096370 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 586/1164] 10.222.0.44 () {58 vars in 1282 bytes} [Wed Jun 29 11:08:22 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 29 11:08:23.102256 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486c2736-66bd-4887-aae7-8687c5c9979e None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:08:23.197096 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486c2736-66bd-4887-aae7-8687c5c9979e None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:08:23.216508 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:08:23.230067 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b30c16a-e9f7-419d-b5a5-7ccf70c42654 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:08:23.230852 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b30c16a-e9f7-419d-b5a5-7ccf70c42654 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:08:23.232133 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b30c16a-e9f7-419d-b5a5-7ccf70c42654 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:08:23.232133 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b30c16a-e9f7-419d-b5a5-7ccf70c42654 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:08:23.238029 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b30c16a-e9f7-419d-b5a5-7ccf70c42654 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:08:23.239445 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b30c16a-e9f7-419d-b5a5-7ccf70c42654 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:08:23.254382 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-8b30c16a-e9f7-419d-b5a5-7ccf70c42654 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 29 11:08:23.254382 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:08:23.254382 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jun 29 11:08:23.254382 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jun 29 11:08:23.254382 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 11:08:23.254382 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 11:08:23.254382 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 11:08:23.254382 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 11:08:23.254382 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jun 29 11:08:23.254382 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:08:23.254382 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 29 11:08:23.254382 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:08:23.254382 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:08:23.254382 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:08:23.254382 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:08:23.254382 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:08:23.254382 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:08:23.255567 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:08:23.255567 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:08:23.255567 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:08:23.255567 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:08:23.255567 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:08:23.255567 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:08:23.255567 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 11:08:23.255567 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 11:08:23.255567 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 11:08:23.255567 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 11:08:23.255567 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:08:23.255567 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:08:23.255567 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:08:23.255567 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:08:23.255567 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:08:23.255567 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:08:23.256508 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:08:23.256508 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jun 29 11:08:23.256508 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:08:23.256508 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:08:23.256508 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:08:23.256508 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:08:23.256508 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:08:23.256508 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:08:23.256508 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:08:23.256508 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:08:23.256508 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jun 29 11:08:23.256508 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 11:08:23.256508 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 29 11:08:23.256508 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:08:23.256508 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 579/1165] 10.222.0.44 () {56 vars in 1117 bytes} [Wed Jun 29 11:08:23 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:08:23.261240 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5a6f49-6b0b-47cf-87ae-8206cb11b8b6 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:08:23.356787 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5a6f49-6b0b-47cf-87ae-8206cb11b8b6 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:08:23.359051 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:08:23.389426 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7013c4-1048-447c-a19f-aa3b93393551 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:08:23.390338 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7013c4-1048-447c-a19f-aa3b93393551 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:08:23.390626 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7013c4-1048-447c-a19f-aa3b93393551 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:08:23.390906 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7013c4-1048-447c-a19f-aa3b93393551 None admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:08:23.391619 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7013c4-1048-447c-a19f-aa3b93393551 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:08:23.391954 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7013c4-1048-447c-a19f-aa3b93393551 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:08:23.393864 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7013c4-1048-447c-a19f-aa3b93393551 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:08:23.400801 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 587/1166] 10.222.0.44 () {56 vars in 1115 bytes} [Wed Jun 29 11:08:23 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:08:25.398198 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423531fb-0952-4809-bfba-fb04b32bbc40 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:08:25.398745 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423531fb-0952-4809-bfba-fb04b32bbc40 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:08:25.398745 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423531fb-0952-4809-bfba-fb04b32bbc40 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:08:25.400161 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 580/1167] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:08:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:08:25.405335 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fa40e2-bf50-42b8-a009-c159187e77fb None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:08:25.405751 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fa40e2-bf50-42b8-a009-c159187e77fb None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:08:25.405903 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fa40e2-bf50-42b8-a009-c159187e77fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:08:25.422194 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-91fa40e2-bf50-42b8-a009-c159187e77fb None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:08:25.423919 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-91fa40e2-bf50-42b8-a009-c159187e77fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:08:25.487084 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 588/1168] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:08:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 11:08:25.510092 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423531fb-0952-4809-bfba-fb04b32bbc40 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:08:25.606114 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423531fb-0952-4809-bfba-fb04b32bbc40 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:08:25.638557 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbea0906-7ded-4e89-9d97-e278bf6bd782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:08:25.639194 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbea0906-7ded-4e89-9d97-e278bf6bd782 None admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:08:25.639475 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbea0906-7ded-4e89-9d97-e278bf6bd782 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:08:25.639745 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbea0906-7ded-4e89-9d97-e278bf6bd782 None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:08:25.640588 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbea0906-7ded-4e89-9d97-e278bf6bd782 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:08:25.641952 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbea0906-7ded-4e89-9d97-e278bf6bd782 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:08:25.717669 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 581/1169] 10.222.0.44 () {60 vars in 1138 bytes} [Wed Jun 29 11:08:25 2022] POST /identity/v3/domains => generated 267 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:08:27.773319 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393f170f-ceba-4723-9e23-2e3a985d076c None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:08:27.773839 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393f170f-ceba-4723-9e23-2e3a985d076c None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:08:27.773930 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393f170f-ceba-4723-9e23-2e3a985d076c None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:08:27.775238 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 589/1170] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:08:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:08:27.780312 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a43463-8932-4421-8f88-032476d6e61b None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:08:27.780696 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a43463-8932-4421-8f88-032476d6e61b None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:08:27.780868 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a43463-8932-4421-8f88-032476d6e61b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:08:27.797150 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-97a43463-8932-4421-8f88-032476d6e61b None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:08:27.798847 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-97a43463-8932-4421-8f88-032476d6e61b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:08:27.832383 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:08:27.864336 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 582/1171] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:08:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 11:08:27.888292 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393f170f-ceba-4723-9e23-2e3a985d076c None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:08:27.982007 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:08:27.986771 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393f170f-ceba-4723-9e23-2e3a985d076c None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:08:28.019766 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a0059e-a750-4840-93cc-ed24d8c2ef61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:08:28.020644 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a0059e-a750-4840-93cc-ed24d8c2ef61 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:08:28.020925 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a0059e-a750-4840-93cc-ed24d8c2ef61 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:08:28.021198 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a0059e-a750-4840-93cc-ed24d8c2ef61 None admin] PATH_INFO: `/v3/domains/c0373804e8a84d23bb3aa88cfdd204b0` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:08:28.031180 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a0059e-a750-4840-93cc-ed24d8c2ef61 None admin] RBAC: Authorizing `identity:get_domain(domain_id=c0373804e8a84d23bb3aa88cfdd204b0)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:08:28.033127 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a0059e-a750-4840-93cc-ed24d8c2ef61 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:08:28.034724 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 590/1172] 10.222.0.44 () {56 vars in 1183 bytes} [Wed Jun 29 11:08:27 2022] GET /identity/v3/domains/c0373804e8a84d23bb3aa88cfdd204b0 => generated 267 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:08:28.040678 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a43463-8932-4421-8f88-032476d6e61b None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:08:28.135805 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a43463-8932-4421-8f88-032476d6e61b None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:08:28.168963 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd234b2e-52b5-4519-ac22-8d9ce470423a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:08:28.169604 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd234b2e-52b5-4519-ac22-8d9ce470423a None admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:08:28.169886 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd234b2e-52b5-4519-ac22-8d9ce470423a None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:08:28.170160 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd234b2e-52b5-4519-ac22-8d9ce470423a None admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:08:28.170871 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd234b2e-52b5-4519-ac22-8d9ce470423a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:08:28.172316 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd234b2e-52b5-4519-ac22-8d9ce470423a None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:08:28.252854 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 583/1173] 10.222.0.44 () {60 vars in 1141 bytes} [Wed Jun 29 11:08:28 2022] POST /identity/v3/projects => generated 361 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:08:30.291643 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7829a58-14eb-4391-a420-30751f307b84 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:08:30.292336 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7829a58-14eb-4391-a420-30751f307b84 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:08:30.292336 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7829a58-14eb-4391-a420-30751f307b84 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:08:30.293576 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 591/1174] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:08:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:08:30.299219 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3c5c6a-a281-4ef7-a89b-6fdd8a1ee758 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:08:30.299625 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3c5c6a-a281-4ef7-a89b-6fdd8a1ee758 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:08:30.299774 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3c5c6a-a281-4ef7-a89b-6fdd8a1ee758 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:08:30.316032 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-3a3c5c6a-a281-4ef7-a89b-6fdd8a1ee758 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:08:30.317230 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-3a3c5c6a-a281-4ef7-a89b-6fdd8a1ee758 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:08:30.333285 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:08:30.381086 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 584/1175] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:08:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 11:08:30.404946 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7829a58-14eb-4391-a420-30751f307b84 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:08:30.502419 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7829a58-14eb-4391-a420-30751f307b84 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:08:30.533988 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410a1273-b91f-4095-b294-28a2c6623915 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:08:30.534820 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410a1273-b91f-4095-b294-28a2c6623915 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:08:30.535100 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410a1273-b91f-4095-b294-28a2c6623915 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:08:30.535371 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410a1273-b91f-4095-b294-28a2c6623915 None admin] PATH_INFO: `/v3/domains/c0373804e8a84d23bb3aa88cfdd204b0` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:08:30.536497 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410a1273-b91f-4095-b294-28a2c6623915 None admin] RBAC: Authorizing `identity:get_domain(domain_id=c0373804e8a84d23bb3aa88cfdd204b0)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:08:30.537900 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410a1273-b91f-4095-b294-28a2c6623915 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:08:30.539451 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 592/1176] 10.222.0.44 () {56 vars in 1183 bytes} [Wed Jun 29 11:08:30 2022] GET /identity/v3/domains/c0373804e8a84d23bb3aa88cfdd204b0 => generated 267 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:08:30.544922 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3c5c6a-a281-4ef7-a89b-6fdd8a1ee758 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:08:30.639751 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3c5c6a-a281-4ef7-a89b-6fdd8a1ee758 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:08:30.672810 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f29666a-2064-4fb5-b063-3791093a9aa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:08:30.673486 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f29666a-2064-4fb5-b063-3791093a9aa5 None admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:08:30.673769 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f29666a-2064-4fb5-b063-3791093a9aa5 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:08:30.674044 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f29666a-2064-4fb5-b063-3791093a9aa5 None admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:08:30.674746 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f29666a-2064-4fb5-b063-3791093a9aa5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:08:30.676177 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f29666a-2064-4fb5-b063-3791093a9aa5 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:08:30.766102 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 585/1177] 10.222.0.44 () {60 vars in 1132 bytes} [Wed Jun 29 11:08:30 2022] POST /identity/v3/users => generated 311 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:08:32.803651 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20192349-4b9d-4f13-b38b-6476399f8ccc None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:08:32.804216 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20192349-4b9d-4f13-b38b-6476399f8ccc None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:08:32.804216 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20192349-4b9d-4f13-b38b-6476399f8ccc None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:08:32.805570 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 593/1178] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:08:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:08:32.810771 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaf7e7d-af18-4f41-ab44-6c72b4ed144d None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:08:32.811173 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaf7e7d-af18-4f41-ab44-6c72b4ed144d None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:08:32.811335 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaf7e7d-af18-4f41-ab44-6c72b4ed144d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:08:32.828385 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-ceaf7e7d-af18-4f41-ab44-6c72b4ed144d None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:08:32.830141 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-ceaf7e7d-af18-4f41-ab44-6c72b4ed144d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:08:32.832073 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:08:32.896193 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 586/1179] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:08:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 11:08:32.920678 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20192349-4b9d-4f13-b38b-6476399f8ccc None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:08:32.949291 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:08:33.019800 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20192349-4b9d-4f13-b38b-6476399f8ccc None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:08:33.051646 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac4b008-0bbd-4231-a9f7-126a11b44435 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:08:33.052456 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac4b008-0bbd-4231-a9f7-126a11b44435 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:08:33.052741 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac4b008-0bbd-4231-a9f7-126a11b44435 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:08:33.053015 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac4b008-0bbd-4231-a9f7-126a11b44435 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:08:33.063184 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac4b008-0bbd-4231-a9f7-126a11b44435 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 11:08:33.071624 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-2ac4b008-0bbd-4231-a9f7-126a11b44435 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 11:08:33.071624 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:08:33.071624 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:08:33.071624 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:08:33.071624 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:08:33.071624 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:08:33.071624 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:08:33.071624 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:08:33.071624 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:08:33.071624 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:08:33.071624 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:08:33.071624 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:08:33.071624 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 11:08:33.071624 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 11:08:33.071624 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 11:08:33.071624 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 11:08:33.072829 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 11:08:33.072829 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 11:08:33.072829 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 11:08:33.072829 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 11:08:33.072829 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:08:33.072829 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:08:33.072829 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:08:33.072829 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:08:33.072829 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:08:33.072829 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:08:33.072829 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:08:33.072829 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jun 29 11:08:33.072829 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:08:33.072829 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:08:33.072829 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:08:33.072829 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:08:33.074066 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:08:33.074066 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:08:33.074066 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:08:33.074066 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:08:33.074066 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 11:08:33.074066 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 11:08:33.074066 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 11:08:33.074066 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 11:08:33.074066 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 11:08:33.074066 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 11:08:33.074066 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 11:08:33.074066 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jun 29 11:08:33.074066 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 594/1180] 10.222.0.44 () {56 vars in 1096 bytes} [Wed Jun 29 11:08:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:08:33.078624 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaf7e7d-af18-4f41-ab44-6c72b4ed144d None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:08:33.172485 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaf7e7d-af18-4f41-ab44-6c72b4ed144d None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:08:33.205999 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb5f4ab-ea64-4353-abc6-e2e30c8a2724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:08:33.206649 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb5f4ab-ea64-4353-abc6-e2e30c8a2724 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:08:33.206930 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb5f4ab-ea64-4353-abc6-e2e30c8a2724 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:08:33.207203 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb5f4ab-ea64-4353-abc6-e2e30c8a2724 None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:08:33.207964 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb5f4ab-ea64-4353-abc6-e2e30c8a2724 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:08:33.208269 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb5f4ab-ea64-4353-abc6-e2e30c8a2724 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:08:33.209621 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb5f4ab-ea64-4353-abc6-e2e30c8a2724 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:08:33.214381 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:08:33.217654 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 587/1181] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Jun 29 11:08:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:08:33.223372 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac4b008-0bbd-4231-a9f7-126a11b44435 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:08:33.305017 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:08:33.318943 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac4b008-0bbd-4231-a9f7-126a11b44435 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:08:33.351234 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fb4596-6cdc-49d6-8386-b997796f4414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:08:33.352103 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fb4596-6cdc-49d6-8386-b997796f4414 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:08:33.352385 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fb4596-6cdc-49d6-8386-b997796f4414 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:08:33.352655 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fb4596-6cdc-49d6-8386-b997796f4414 None admin] PATH_INFO: `/v3/users/0e086f52d23c4e59b849a4cef533b964` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:08:33.365384 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fb4596-6cdc-49d6-8386-b997796f4414 None admin] RBAC: Authorizing `identity:get_user(user_id=0e086f52d23c4e59b849a4cef533b964)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:08:33.366799 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fb4596-6cdc-49d6-8386-b997796f4414 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:08:33.368464 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 595/1182] 10.222.0.44 () {56 vars in 1177 bytes} [Wed Jun 29 11:08:33 2022] GET /identity/v3/users/0e086f52d23c4e59b849a4cef533b964 => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:08:33.374278 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb5f4ab-ea64-4353-abc6-e2e30c8a2724 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:08:33.469189 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb5f4ab-ea64-4353-abc6-e2e30c8a2724 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:08:33.501589 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c510c0-0fd3-46f2-b945-7692dfa11025 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:08:33.502219 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c510c0-0fd3-46f2-b945-7692dfa11025 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:08:33.502500 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c510c0-0fd3-46f2-b945-7692dfa11025 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:08:33.502774 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c510c0-0fd3-46f2-b945-7692dfa11025 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:08:33.564460 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:08:33.566673 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c510c0-0fd3-46f2-b945-7692dfa11025 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:08:33.568133 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c510c0-0fd3-46f2-b945-7692dfa11025 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:08:33.662119 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 588/1183] 10.222.0.44 () {58 vars in 1282 bytes} [Wed Jun 29 11:08:33 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 29 11:08:33.668014 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fb4596-6cdc-49d6-8386-b997796f4414 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:08:33.763964 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fb4596-6cdc-49d6-8386-b997796f4414 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:08:33.796312 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff3fe22-5e0d-4cd1-8f19-c82a86c27cd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:08:33.797151 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff3fe22-5e0d-4cd1-8f19-c82a86c27cd5 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:08:33.797432 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff3fe22-5e0d-4cd1-8f19-c82a86c27cd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:08:33.797708 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff3fe22-5e0d-4cd1-8f19-c82a86c27cd5 None admin] PATH_INFO: `/v3/projects/cc68bd6d2ca7414b910c37687f631a78` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:08:33.798891 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff3fe22-5e0d-4cd1-8f19-c82a86c27cd5 None admin] RBAC: Authorizing `identity:get_project(project_id=cc68bd6d2ca7414b910c37687f631a78)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:08:33.800348 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff3fe22-5e0d-4cd1-8f19-c82a86c27cd5 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:08:33.801991 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 596/1184] 10.222.0.44 () {56 vars in 1186 bytes} [Wed Jun 29 11:08:33 2022] GET /identity/v3/projects/cc68bd6d2ca7414b910c37687f631a78 => generated 361 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:08:33.807808 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c510c0-0fd3-46f2-b945-7692dfa11025 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:08:33.899088 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:08:33.901919 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c510c0-0fd3-46f2-b945-7692dfa11025 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:08:33.935893 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f3c0b1-8174-437b-88e2-abf1492478e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:08:33.936554 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f3c0b1-8174-437b-88e2-abf1492478e6 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:08:33.936838 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f3c0b1-8174-437b-88e2-abf1492478e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:08:33.937115 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f3c0b1-8174-437b-88e2-abf1492478e6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:08:33.937939 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f3c0b1-8174-437b-88e2-abf1492478e6 None admin] RBAC: Adding query filter params (role.id=e95c2ce63e114cc19a9dda6ad21e794f, scope.project.id=cc68bd6d2ca7414b910c37687f631a78, user.id=0e086f52d23c4e59b849a4cef533b964) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:08:33.938252 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f3c0b1-8174-437b-88e2-abf1492478e6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:08:33.939709 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f3c0b1-8174-437b-88e2-abf1492478e6 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:08:33.945073 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 589/1185] 10.222.0.44 () {56 vars in 1374 bytes} [Wed Jun 29 11:08:33 2022] GET /identity/v3/role_assignments?user.id=0e086f52d23c4e59b849a4cef533b964&scope.project.id=cc68bd6d2ca7414b910c37687f631a78&role.id=e95c2ce63e114cc19a9dda6ad21e794f => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:08:35.920907 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8869e5-3b4d-4f2a-9e0b-cfcb9d78222e None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:08:35.921349 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8869e5-3b4d-4f2a-9e0b-cfcb9d78222e None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:08:35.921452 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8869e5-3b4d-4f2a-9e0b-cfcb9d78222e None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:08:35.922820 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 597/1186] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:08:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:08:35.928074 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c10ea3b-fa6e-4343-bcca-f224c023e36c None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:08:35.928494 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c10ea3b-fa6e-4343-bcca-f224c023e36c None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:08:35.928622 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c10ea3b-fa6e-4343-bcca-f224c023e36c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:08:35.945065 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-4c10ea3b-fa6e-4343-bcca-f224c023e36c None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:08:35.946515 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-4c10ea3b-fa6e-4343-bcca-f224c023e36c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:08:36.009928 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 590/1187] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:08:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 11:08:36.032983 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8869e5-3b4d-4f2a-9e0b-cfcb9d78222e None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:08:36.052747 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:08:36.146289 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8869e5-3b4d-4f2a-9e0b-cfcb9d78222e None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:08:36.178011 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275b7711-5252-4c06-b560-f1c1b9f0a1f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:08:36.178722 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275b7711-5252-4c06-b560-f1c1b9f0a1f6 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:08:36.179000 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275b7711-5252-4c06-b560-f1c1b9f0a1f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:08:36.179268 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275b7711-5252-4c06-b560-f1c1b9f0a1f6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:08:36.189433 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275b7711-5252-4c06-b560-f1c1b9f0a1f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 11:08:36.197259 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-275b7711-5252-4c06-b560-f1c1b9f0a1f6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 11:08:36.197259 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:08:36.197259 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:08:36.197259 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:08:36.197259 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:08:36.197259 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:08:36.197259 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:08:36.197259 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:08:36.197259 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:08:36.197259 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:08:36.197259 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:08:36.197259 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:08:36.197259 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 11:08:36.197259 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 11:08:36.197259 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 11:08:36.197259 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 11:08:36.198356 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 11:08:36.198356 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 11:08:36.198356 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 11:08:36.198356 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 11:08:36.198356 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:08:36.198356 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:08:36.198356 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:08:36.198356 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:08:36.198356 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:08:36.198356 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:08:36.198356 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:08:36.198356 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jun 29 11:08:36.198356 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:08:36.198356 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:08:36.198356 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:08:36.198356 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:08:36.199492 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:08:36.199492 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:08:36.199492 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:08:36.199492 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:08:36.199492 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 11:08:36.199492 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 11:08:36.199492 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 11:08:36.199492 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 11:08:36.199492 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 11:08:36.199492 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 11:08:36.199492 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 11:08:36.199492 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jun 29 11:08:36.199492 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 598/1188] 10.222.0.44 () {56 vars in 1096 bytes} [Wed Jun 29 11:08:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:08:36.205283 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c10ea3b-fa6e-4343-bcca-f224c023e36c None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:08:36.294945 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:08:36.301481 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c10ea3b-fa6e-4343-bcca-f224c023e36c None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:08:36.335186 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be27fce0-b33b-45aa-9a2b-34d0da321744 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:08:36.335861 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be27fce0-b33b-45aa-9a2b-34d0da321744 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:08:36.336156 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be27fce0-b33b-45aa-9a2b-34d0da321744 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:08:36.336433 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be27fce0-b33b-45aa-9a2b-34d0da321744 None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:08:36.337177 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be27fce0-b33b-45aa-9a2b-34d0da321744 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:08:36.337486 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be27fce0-b33b-45aa-9a2b-34d0da321744 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:08:36.338830 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be27fce0-b33b-45aa-9a2b-34d0da321744 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:08:36.346679 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 591/1189] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Jun 29 11:08:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:08:36.352639 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275b7711-5252-4c06-b560-f1c1b9f0a1f6 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:08:36.422165 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:08:36.451582 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275b7711-5252-4c06-b560-f1c1b9f0a1f6 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:08:36.484763 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c415bd59-71c8-4927-bb79-981ab4911295 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:08:36.485650 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c415bd59-71c8-4927-bb79-981ab4911295 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:08:36.485935 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c415bd59-71c8-4927-bb79-981ab4911295 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:08:36.486213 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c415bd59-71c8-4927-bb79-981ab4911295 None admin] PATH_INFO: `/v3/users/0e086f52d23c4e59b849a4cef533b964` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:08:36.487387 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c415bd59-71c8-4927-bb79-981ab4911295 None admin] RBAC: Authorizing `identity:get_user(user_id=0e086f52d23c4e59b849a4cef533b964)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:08:36.488839 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c415bd59-71c8-4927-bb79-981ab4911295 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:08:36.490450 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 599/1190] 10.222.0.44 () {56 vars in 1177 bytes} [Wed Jun 29 11:08:36 2022] GET /identity/v3/users/0e086f52d23c4e59b849a4cef533b964 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:08:36.496440 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be27fce0-b33b-45aa-9a2b-34d0da321744 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:08:36.593016 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be27fce0-b33b-45aa-9a2b-34d0da321744 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:08:36.625386 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bbcddb-2ccb-4a9e-8636-dfb6c64dec0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:08:36.626277 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bbcddb-2ccb-4a9e-8636-dfb6c64dec0c None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:08:36.626561 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bbcddb-2ccb-4a9e-8636-dfb6c64dec0c None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:08:36.626839 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bbcddb-2ccb-4a9e-8636-dfb6c64dec0c None admin] PATH_INFO: `/v3/projects/cc68bd6d2ca7414b910c37687f631a78` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:08:36.628032 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bbcddb-2ccb-4a9e-8636-dfb6c64dec0c None admin] RBAC: Authorizing `identity:get_project(project_id=cc68bd6d2ca7414b910c37687f631a78)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:08:36.629449 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bbcddb-2ccb-4a9e-8636-dfb6c64dec0c None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:08:36.631123 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 592/1191] 10.222.0.44 () {56 vars in 1186 bytes} [Wed Jun 29 11:08:36 2022] GET /identity/v3/projects/cc68bd6d2ca7414b910c37687f631a78 => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:08:36.636801 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c415bd59-71c8-4927-bb79-981ab4911295 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:08:36.732423 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c415bd59-71c8-4927-bb79-981ab4911295 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:08:36.764375 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0908e4-ff8c-4a21-ad33-3a3b6b4bca09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:08:36.765240 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0908e4-ff8c-4a21-ad33-3a3b6b4bca09 None admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:08:36.765523 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0908e4-ff8c-4a21-ad33-3a3b6b4bca09 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:08:36.765801 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0908e4-ff8c-4a21-ad33-3a3b6b4bca09 None admin] PATH_INFO: `/v3/projects/cc68bd6d2ca7414b910c37687f631a78/users/0e086f52d23c4e59b849a4cef533b964/roles/e95c2ce63e114cc19a9dda6ad21e794f` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:08:36.768284 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0908e4-ff8c-4a21-ad33-3a3b6b4bca09 None admin] RBAC: Authorizing `identity:create_grant(project_id=cc68bd6d2ca7414b910c37687f631a78, user_id=0e086f52d23c4e59b849a4cef533b964, role_id=e95c2ce63e114cc19a9dda6ad21e794f)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:08:36.769706 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0908e4-ff8c-4a21-ad33-3a3b6b4bca09 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:08:36.833704 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:08:36.838213 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 600/1192] 10.222.0.44 () {58 vars in 1439 bytes} [Wed Jun 29 11:08:36 2022] PUT /identity/v3/projects/cc68bd6d2ca7414b910c37687f631a78/users/0e086f52d23c4e59b849a4cef533b964/roles/e95c2ce63e114cc19a9dda6ad21e794f => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:08:38.807085 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befd9cdb-6a01-4832-a9e0-5c973db4065c None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:08:38.807661 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befd9cdb-6a01-4832-a9e0-5c973db4065c None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:08:38.807661 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befd9cdb-6a01-4832-a9e0-5c973db4065c None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:08:38.809032 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 593/1193] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:08:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:08:38.814639 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5ee89d-4440-410c-87bb-0170963de068 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:08:38.815023 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5ee89d-4440-410c-87bb-0170963de068 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:08:38.815184 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5ee89d-4440-410c-87bb-0170963de068 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:08:38.835849 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-2b5ee89d-4440-410c-87bb-0170963de068 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:08:38.837436 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-2b5ee89d-4440-410c-87bb-0170963de068 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:08:38.907356 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 601/1194] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:08:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 11:08:38.930915 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befd9cdb-6a01-4832-a9e0-5c973db4065c None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:08:38.956647 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:08:39.029992 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befd9cdb-6a01-4832-a9e0-5c973db4065c None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:08:39.062937 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5547cad-65e1-4848-9a79-1907eb1cd2b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:08:39.064245 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5547cad-65e1-4848-9a79-1907eb1cd2b9 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:08:39.064527 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5547cad-65e1-4848-9a79-1907eb1cd2b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:08:39.064800 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5547cad-65e1-4848-9a79-1907eb1cd2b9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:08:39.074675 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5547cad-65e1-4848-9a79-1907eb1cd2b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 11:08:39.082253 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-c5547cad-65e1-4848-9a79-1907eb1cd2b9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 11:08:39.082253 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:08:39.082253 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:08:39.082253 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:08:39.082253 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:08:39.082253 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:08:39.082253 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:08:39.082253 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:08:39.082253 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:08:39.082253 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:08:39.082253 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:08:39.082253 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:08:39.082253 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 11:08:39.082253 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 11:08:39.082253 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 11:08:39.082253 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 11:08:39.083363 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 11:08:39.083363 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 11:08:39.083363 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 11:08:39.083363 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 11:08:39.083363 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:08:39.083363 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:08:39.083363 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:08:39.083363 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:08:39.083363 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:08:39.083363 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:08:39.083363 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:08:39.083363 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jun 29 11:08:39.083363 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:08:39.083363 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:08:39.083363 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:08:39.083363 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:08:39.084353 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:08:39.084353 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:08:39.084353 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:08:39.084353 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:08:39.084353 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 11:08:39.084353 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 11:08:39.084353 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 11:08:39.084353 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 11:08:39.084353 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 11:08:39.084353 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 11:08:39.084353 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 11:08:39.084353 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:08:39.084353 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 594/1195] 10.222.0.44 () {56 vars in 1096 bytes} [Wed Jun 29 11:08:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:08:39.089436 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5ee89d-4440-410c-87bb-0170963de068 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:08:39.183386 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5ee89d-4440-410c-87bb-0170963de068 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:08:39.210809 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:08:39.216344 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c24215-f39c-42a2-a406-e58f590cff57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:08:39.222236 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c24215-f39c-42a2-a406-e58f590cff57 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:08:39.222236 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c24215-f39c-42a2-a406-e58f590cff57 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:08:39.222236 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c24215-f39c-42a2-a406-e58f590cff57 None admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:08:39.222236 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c24215-f39c-42a2-a406-e58f590cff57 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:08:39.222236 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c24215-f39c-42a2-a406-e58f590cff57 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:08:39.222236 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c24215-f39c-42a2-a406-e58f590cff57 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:08:39.227641 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 602/1196] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Jun 29 11:08:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:08:39.233156 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5547cad-65e1-4848-9a79-1907eb1cd2b9 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:08:39.314641 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:08:39.329701 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5547cad-65e1-4848-9a79-1907eb1cd2b9 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:08:39.363473 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a77eec8-b636-483d-a5b7-2fc66d74c027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:08:39.364387 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a77eec8-b636-483d-a5b7-2fc66d74c027 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:08:39.364678 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a77eec8-b636-483d-a5b7-2fc66d74c027 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:08:39.364959 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a77eec8-b636-483d-a5b7-2fc66d74c027 None admin] PATH_INFO: `/v3/users/0e086f52d23c4e59b849a4cef533b964` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:08:39.366133 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a77eec8-b636-483d-a5b7-2fc66d74c027 None admin] RBAC: Authorizing `identity:get_user(user_id=0e086f52d23c4e59b849a4cef533b964)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:08:39.367562 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a77eec8-b636-483d-a5b7-2fc66d74c027 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:08:39.369212 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 595/1197] 10.222.0.44 () {56 vars in 1177 bytes} [Wed Jun 29 11:08:39 2022] GET /identity/v3/users/0e086f52d23c4e59b849a4cef533b964 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:08:39.374878 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c24215-f39c-42a2-a406-e58f590cff57 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:08:39.471125 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c24215-f39c-42a2-a406-e58f590cff57 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:08:39.503117 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515b2e60-d4ad-41c0-9c92-1242be492343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:08:39.503749 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515b2e60-d4ad-41c0-9c92-1242be492343 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:08:39.504060 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515b2e60-d4ad-41c0-9c92-1242be492343 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:08:39.504336 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515b2e60-d4ad-41c0-9c92-1242be492343 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:08:39.559409 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:08:39.571288 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515b2e60-d4ad-41c0-9c92-1242be492343 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:08:39.571288 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515b2e60-d4ad-41c0-9c92-1242be492343 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:08:39.662885 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 603/1198] 10.222.0.44 () {58 vars in 1282 bytes} [Wed Jun 29 11:08:39 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 29 11:08:39.668671 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a77eec8-b636-483d-a5b7-2fc66d74c027 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:08:39.765991 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a77eec8-b636-483d-a5b7-2fc66d74c027 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:08:39.798859 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649b2240-87ef-4fde-a4fc-f6f85b87e9c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:08:39.799694 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649b2240-87ef-4fde-a4fc-f6f85b87e9c5 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:08:39.800005 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649b2240-87ef-4fde-a4fc-f6f85b87e9c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:08:39.800279 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649b2240-87ef-4fde-a4fc-f6f85b87e9c5 None admin] PATH_INFO: `/v3/projects/cc68bd6d2ca7414b910c37687f631a78` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:08:39.801434 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649b2240-87ef-4fde-a4fc-f6f85b87e9c5 None admin] RBAC: Authorizing `identity:get_project(project_id=cc68bd6d2ca7414b910c37687f631a78)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:08:39.802834 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649b2240-87ef-4fde-a4fc-f6f85b87e9c5 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:08:39.804544 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 596/1199] 10.222.0.44 () {56 vars in 1186 bytes} [Wed Jun 29 11:08:39 2022] GET /identity/v3/projects/cc68bd6d2ca7414b910c37687f631a78 => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:08:39.810339 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515b2e60-d4ad-41c0-9c92-1242be492343 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:08:39.900698 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:08:39.908945 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515b2e60-d4ad-41c0-9c92-1242be492343 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:08:39.942934 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49471021-a5c1-47bb-981e-8b1a1a2ac319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:08:39.943586 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49471021-a5c1-47bb-981e-8b1a1a2ac319 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:08:39.943905 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49471021-a5c1-47bb-981e-8b1a1a2ac319 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:08:39.944190 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49471021-a5c1-47bb-981e-8b1a1a2ac319 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:08:39.945047 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49471021-a5c1-47bb-981e-8b1a1a2ac319 None admin] RBAC: Adding query filter params (role.id=e95c2ce63e114cc19a9dda6ad21e794f, scope.project.id=cc68bd6d2ca7414b910c37687f631a78, user.id=0e086f52d23c4e59b849a4cef533b964) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:08:39.945362 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49471021-a5c1-47bb-981e-8b1a1a2ac319 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:08:39.947060 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49471021-a5c1-47bb-981e-8b1a1a2ac319 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:08:39.953037 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 604/1200] 10.222.0.44 () {56 vars in 1374 bytes} [Wed Jun 29 11:08:39 2022] GET /identity/v3/role_assignments?user.id=0e086f52d23c4e59b849a4cef533b964&scope.project.id=cc68bd6d2ca7414b910c37687f631a78&role.id=e95c2ce63e114cc19a9dda6ad21e794f => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:08:49.794844 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=90 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:08:49.950751 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=22 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:09:53.235154 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b76715-cb99-47af-8ca3-bdf0eaeebc19 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:09:53.235721 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b76715-cb99-47af-8ca3-bdf0eaeebc19 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:09:53.235721 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b76715-cb99-47af-8ca3-bdf0eaeebc19 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:09:53.237115 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 597/1201] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:09:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:09:53.242403 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aec0248-4ff1-4c26-9970-07ba6c065482 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:09:53.243133 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aec0248-4ff1-4c26-9970-07ba6c065482 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:09:53.243133 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aec0248-4ff1-4c26-9970-07ba6c065482 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:09:53.259801 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-3aec0248-4ff1-4c26-9970-07ba6c065482 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:09:53.260993 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-3aec0248-4ff1-4c26-9970-07ba6c065482 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:09:53.339371 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 605/1202] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:09:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 11:09:53.369396 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b76715-cb99-47af-8ca3-bdf0eaeebc19 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:09:53.477419 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b76715-cb99-47af-8ca3-bdf0eaeebc19 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:09:53.511597 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd43e32c-b3dd-4212-b2f1-ed054bf19d1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:09:53.512338 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd43e32c-b3dd-4212-b2f1-ed054bf19d1d None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:09:53.512620 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd43e32c-b3dd-4212-b2f1-ed054bf19d1d None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:09:53.512890 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd43e32c-b3dd-4212-b2f1-ed054bf19d1d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:09:53.522893 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd43e32c-b3dd-4212-b2f1-ed054bf19d1d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:09:53.526710 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd43e32c-b3dd-4212-b2f1-ed054bf19d1d None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:09:53.541975 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-bd43e32c-b3dd-4212-b2f1-ed054bf19d1d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 11:09:53.541975 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:09:53.541975 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:09:53.541975 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:09:53.541975 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:09:53.541975 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:09:53.541975 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:09:53.541975 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:09:53.541975 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:09:53.541975 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:09:53.541975 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:09:53.541975 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:09:53.541975 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 11:09:53.541975 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 11:09:53.541975 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 11:09:53.546182 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 11:09:53.546182 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:09:53.546182 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:09:53.546182 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:09:53.546182 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:09:53.546182 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:09:53.546182 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:09:53.546182 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:09:53.546182 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jun 29 11:09:53.546182 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:09:53.546182 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:09:53.546182 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:09:53.546182 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:09:53.546182 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:09:53.546182 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:09:53.546182 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:09:53.546182 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:09:53.547382 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 11:09:53.547382 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 11:09:53.547382 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 11:09:53.547382 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:09:53.547382 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 598/1203] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jun 29 11:09:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:09:53.549237 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aec0248-4ff1-4c26-9970-07ba6c065482 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:09:53.646435 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aec0248-4ff1-4c26-9970-07ba6c065482 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:09:53.674655 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:09:53.680958 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ec1d17-2414-472e-a693-fa33b503e960 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:09:53.681623 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ec1d17-2414-472e-a693-fa33b503e960 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:09:53.681905 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ec1d17-2414-472e-a693-fa33b503e960 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:09:53.682184 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ec1d17-2414-472e-a693-fa33b503e960 None admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:09:53.682957 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ec1d17-2414-472e-a693-fa33b503e960 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:09:53.683263 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ec1d17-2414-472e-a693-fa33b503e960 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:09:53.684674 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ec1d17-2414-472e-a693-fa33b503e960 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:09:53.696686 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 606/1204] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Jun 29 11:09:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:09:53.702507 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd43e32c-b3dd-4212-b2f1-ed054bf19d1d None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:09:53.755026 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:09:53.800076 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd43e32c-b3dd-4212-b2f1-ed054bf19d1d None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:09:53.833658 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2e55f3-37ea-489c-9dd8-78785ec22585 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:09:53.834320 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2e55f3-37ea-489c-9dd8-78785ec22585 None admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:09:53.834606 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2e55f3-37ea-489c-9dd8-78785ec22585 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:09:53.834886 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2e55f3-37ea-489c-9dd8-78785ec22585 None admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:09:53.835598 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2e55f3-37ea-489c-9dd8-78785ec22585 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:09:53.837047 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2e55f3-37ea-489c-9dd8-78785ec22585 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:09:53.900595 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 599/1205] 10.222.0.44 () {60 vars in 1132 bytes} [Wed Jun 29 11:09:53 2022] POST /identity/v3/users => generated 251 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:09:55.967217 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5fd7cc-0226-48ee-9cd2-52b935d6fd53 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:09:55.967746 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5fd7cc-0226-48ee-9cd2-52b935d6fd53 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:09:55.967874 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5fd7cc-0226-48ee-9cd2-52b935d6fd53 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:09:55.969225 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 607/1206] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:09:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:09:55.974486 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c04642-8b69-4da9-bd60-3d7bdf088bf6 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:09:55.974890 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c04642-8b69-4da9-bd60-3d7bdf088bf6 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:09:55.977936 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c04642-8b69-4da9-bd60-3d7bdf088bf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:09:55.991805 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-c8c04642-8b69-4da9-bd60-3d7bdf088bf6 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:09:55.993045 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-c8c04642-8b69-4da9-bd60-3d7bdf088bf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:09:56.061296 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 600/1207] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:09:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 11:09:56.088128 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5fd7cc-0226-48ee-9cd2-52b935d6fd53 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:09:56.188411 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5fd7cc-0226-48ee-9cd2-52b935d6fd53 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:09:56.221716 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb96901d-2c90-4d55-8756-0937449ea97b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:09:56.222521 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb96901d-2c90-4d55-8756-0937449ea97b None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:09:56.222806 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb96901d-2c90-4d55-8756-0937449ea97b None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:09:56.223079 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb96901d-2c90-4d55-8756-0937449ea97b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:09:56.235984 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb96901d-2c90-4d55-8756-0937449ea97b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 11:09:56.243956 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-cb96901d-2c90-4d55-8756-0937449ea97b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 29 11:09:56.243956 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:09:56.243956 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:09:56.243956 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:09:56.243956 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:09:56.243956 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:09:56.243956 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:09:56.243956 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:09:56.243956 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:09:56.243956 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:09:56.243956 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:09:56.243956 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:09:56.243956 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 11:09:56.243956 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 11:09:56.243956 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 11:09:56.243956 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 11:09:56.245122 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 11:09:56.245122 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 11:09:56.245122 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 11:09:56.245122 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 11:09:56.245122 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:09:56.245122 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:09:56.245122 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:09:56.245122 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:09:56.245122 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:09:56.245122 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:09:56.245122 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:09:56.245122 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jun 29 11:09:56.245122 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:09:56.245122 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:09:56.245122 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:09:56.245122 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:09:56.246305 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:09:56.246305 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:09:56.246305 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:09:56.246305 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:09:56.246305 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 11:09:56.246305 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 11:09:56.246305 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 11:09:56.246305 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 11:09:56.246305 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 11:09:56.246305 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 11:09:56.246305 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 29 11:09:56.246305 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jun 29 11:09:56.246305 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 608/1208] 10.222.0.44 () {56 vars in 1102 bytes} [Wed Jun 29 11:09:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:09:56.251163 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c04642-8b69-4da9-bd60-3d7bdf088bf6 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:09:56.290193 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:09:56.347628 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c04642-8b69-4da9-bd60-3d7bdf088bf6 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:09:56.382166 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19d6966-bf0d-4b36-a7e8-78d13d74332a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:09:56.382821 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19d6966-bf0d-4b36-a7e8-78d13d74332a None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:09:56.383100 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19d6966-bf0d-4b36-a7e8-78d13d74332a None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:09:56.383369 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19d6966-bf0d-4b36-a7e8-78d13d74332a None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:09:56.384125 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19d6966-bf0d-4b36-a7e8-78d13d74332a None admin] RBAC: Adding query filter params (name=service) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:09:56.384427 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19d6966-bf0d-4b36-a7e8-78d13d74332a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:09:56.385811 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19d6966-bf0d-4b36-a7e8-78d13d74332a None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:09:56.393406 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 601/1209] 10.222.0.44 () {56 vars in 1103 bytes} [Wed Jun 29 11:09:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:09:56.399672 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb96901d-2c90-4d55-8756-0937449ea97b None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:09:56.450309 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:09:56.496471 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb96901d-2c90-4d55-8756-0937449ea97b None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:09:56.529266 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84845d69-abe6-4572-b1c5-0729e5458a82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:09:56.530051 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84845d69-abe6-4572-b1c5-0729e5458a82 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:09:56.530337 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84845d69-abe6-4572-b1c5-0729e5458a82 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:09:56.530614 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84845d69-abe6-4572-b1c5-0729e5458a82 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:09:56.540240 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84845d69-abe6-4572-b1c5-0729e5458a82 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:09:56.541671 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84845d69-abe6-4572-b1c5-0729e5458a82 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:09:56.559861 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-84845d69-abe6-4572-b1c5-0729e5458a82 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 11:09:56.559861 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:09:56.559861 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:09:56.559861 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:09:56.559861 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:09:56.559861 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:09:56.559861 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:09:56.559861 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:09:56.559861 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:09:56.559861 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:09:56.559861 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:09:56.559861 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:09:56.559861 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 11:09:56.559861 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 11:09:56.559861 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 11:09:56.563728 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 11:09:56.563728 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:09:56.563728 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:09:56.563728 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:09:56.563728 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:09:56.563728 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:09:56.563728 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:09:56.563728 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:09:56.563728 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jun 29 11:09:56.563728 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:09:56.563728 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:09:56.563728 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:09:56.563728 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:09:56.563728 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:09:56.563728 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:09:56.563728 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:09:56.563728 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:09:56.564890 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 11:09:56.564890 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 11:09:56.564890 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 11:09:56.564890 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jun 29 11:09:56.564890 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 609/1210] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jun 29 11:09:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:09:56.567408 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19d6966-bf0d-4b36-a7e8-78d13d74332a None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:09:56.662875 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:09:56.665612 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19d6966-bf0d-4b36-a7e8-78d13d74332a None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:09:56.699264 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56af0be-15a2-4176-91d5-e64ffd99f06d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:09:56.700000 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56af0be-15a2-4176-91d5-e64ffd99f06d None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:09:56.700269 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56af0be-15a2-4176-91d5-e64ffd99f06d None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:09:56.700588 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56af0be-15a2-4176-91d5-e64ffd99f06d None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:09:56.701254 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56af0be-15a2-4176-91d5-e64ffd99f06d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:09:56.701559 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56af0be-15a2-4176-91d5-e64ffd99f06d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:09:56.702983 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56af0be-15a2-4176-91d5-e64ffd99f06d None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:09:56.713786 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 602/1211] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Jun 29 11:09:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:09:56.719432 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84845d69-abe6-4572-b1c5-0729e5458a82 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:09:56.817341 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84845d69-abe6-4572-b1c5-0729e5458a82 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:09:56.818864 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:09:56.850300 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3ceb65-0c34-4939-ba87-519b027a400c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:09:56.851077 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3ceb65-0c34-4939-ba87-519b027a400c None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:09:56.851357 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3ceb65-0c34-4939-ba87-519b027a400c None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:09:56.851632 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3ceb65-0c34-4939-ba87-519b027a400c None admin] PATH_INFO: `/v3/users/placement` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:09:56.858360 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3ceb65-0c34-4939-ba87-519b027a400c None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:09:56.859873 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3ceb65-0c34-4939-ba87-519b027a400c None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:09:56.875036 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-5e3ceb65-0c34-4939-ba87-519b027a400c None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 29 11:09:56.875036 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:09:56.875036 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:09:56.875036 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:09:56.875036 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:09:56.875036 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:09:56.875036 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:09:56.875036 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:09:56.875036 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:09:56.875036 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:09:56.875036 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:09:56.875036 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:09:56.875036 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 11:09:56.875036 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 11:09:56.875036 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 11:09:56.875036 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 11:09:56.876232 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:09:56.876232 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:09:56.876232 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 11:09:56.876232 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 11:09:56.876232 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 11:09:56.876232 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 11:09:56.876232 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:09:56.876232 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:09:56.876232 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:09:56.876232 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:09:56.876232 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:09:56.876232 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jun 29 11:09:56.876232 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:09:56.876232 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:09:56.876232 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:09:56.876232 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:09:56.876232 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:09:56.876232 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:09:56.877375 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:09:56.877375 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:09:56.877375 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 11:09:56.877375 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 11:09:56.877375 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 11:09:56.877375 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 11:09:56.877375 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 11:09:56.877375 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 11:09:56.877375 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 29 11:09:56.877375 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jun 29 11:09:56.877375 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 610/1212] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jun 29 11:09:56 2022] GET /identity/v3/users/placement => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:09:56.882118 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56af0be-15a2-4176-91d5-e64ffd99f06d None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:09:56.981186 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56af0be-15a2-4176-91d5-e64ffd99f06d None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:09:57.013297 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee57d72-9ca8-4263-b416-9fabd063c5c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:09:57.013944 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee57d72-9ca8-4263-b416-9fabd063c5c2 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:09:57.014223 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee57d72-9ca8-4263-b416-9fabd063c5c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:09:57.014497 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee57d72-9ca8-4263-b416-9fabd063c5c2 None admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:09:57.015315 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee57d72-9ca8-4263-b416-9fabd063c5c2 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:09:57.015621 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee57d72-9ca8-4263-b416-9fabd063c5c2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:09:57.017059 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee57d72-9ca8-4263-b416-9fabd063c5c2 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:09:57.033552 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 603/1213] 10.222.0.44 () {56 vars in 1143 bytes} [Wed Jun 29 11:09:56 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:09:57.039854 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3ceb65-0c34-4939-ba87-519b027a400c None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:09:57.138052 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3ceb65-0c34-4939-ba87-519b027a400c None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:09:57.171630 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c108f18-736f-45c2-98a9-77d366904d65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:09:57.172295 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c108f18-736f-45c2-98a9-77d366904d65 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:09:57.172574 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c108f18-736f-45c2-98a9-77d366904d65 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:09:57.172846 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c108f18-736f-45c2-98a9-77d366904d65 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:09:57.202079 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:09:57.239083 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c108f18-736f-45c2-98a9-77d366904d65 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:09:57.240476 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c108f18-736f-45c2-98a9-77d366904d65 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:09:57.339818 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 611/1214] 10.222.0.44 () {58 vars in 1282 bytes} [Wed Jun 29 11:09:57 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 29 11:09:57.345822 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee57d72-9ca8-4263-b416-9fabd063c5c2 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:09:57.365566 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:09:57.445152 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee57d72-9ca8-4263-b416-9fabd063c5c2 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:09:57.478854 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdd75fc-0234-4b14-bf1e-0ae16ab8e204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:09:57.479636 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdd75fc-0234-4b14-bf1e-0ae16ab8e204 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:09:57.479946 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdd75fc-0234-4b14-bf1e-0ae16ab8e204 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:09:57.480218 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdd75fc-0234-4b14-bf1e-0ae16ab8e204 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:09:57.489903 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdd75fc-0234-4b14-bf1e-0ae16ab8e204 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:09:57.491268 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdd75fc-0234-4b14-bf1e-0ae16ab8e204 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:09:57.509217 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-9cdd75fc-0234-4b14-bf1e-0ae16ab8e204 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 11:09:57.509217 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:09:57.509217 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:09:57.509217 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:09:57.509217 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:09:57.509217 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:09:57.509217 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:09:57.509217 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:09:57.509217 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:09:57.509217 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:09:57.509217 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:09:57.509217 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:09:57.509217 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 11:09:57.509217 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 11:09:57.509217 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 11:09:57.510317 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 11:09:57.510317 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:09:57.510317 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:09:57.510317 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:09:57.510317 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:09:57.510317 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:09:57.510317 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:09:57.510317 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:09:57.510317 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jun 29 11:09:57.510317 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:09:57.510317 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:09:57.510317 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:09:57.510317 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:09:57.510317 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:09:57.510317 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:09:57.510317 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:09:57.510317 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:09:57.511495 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 11:09:57.511495 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 11:09:57.511495 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 11:09:57.511495 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:09:57.511495 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 604/1215] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jun 29 11:09:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 11:09:57.516321 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c108f18-736f-45c2-98a9-77d366904d65 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:09:57.579386 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:09:57.616094 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c108f18-736f-45c2-98a9-77d366904d65 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:09:57.649319 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d428a7a-c8d2-4682-84bd-8f3abd25a677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:09:57.662399 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d428a7a-c8d2-4682-84bd-8f3abd25a677 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:09:57.662399 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d428a7a-c8d2-4682-84bd-8f3abd25a677 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:09:57.662399 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d428a7a-c8d2-4682-84bd-8f3abd25a677 None admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:09:57.662399 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d428a7a-c8d2-4682-84bd-8f3abd25a677 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:09:57.662399 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d428a7a-c8d2-4682-84bd-8f3abd25a677 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:09:57.662399 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d428a7a-c8d2-4682-84bd-8f3abd25a677 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:09:57.663669 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 612/1216] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Jun 29 11:09:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:09:57.669343 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdd75fc-0234-4b14-bf1e-0ae16ab8e204 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:09:57.745744 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:09:57.777267 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdd75fc-0234-4b14-bf1e-0ae16ab8e204 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:09:57.810410 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf46c7c-8e02-4041-800b-126f241b1972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:09:57.811195 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf46c7c-8e02-4041-800b-126f241b1972 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:09:57.811484 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf46c7c-8e02-4041-800b-126f241b1972 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:09:57.811763 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf46c7c-8e02-4041-800b-126f241b1972 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:09:57.818825 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf46c7c-8e02-4041-800b-126f241b1972 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:09:57.820320 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf46c7c-8e02-4041-800b-126f241b1972 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:09:57.834715 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-0cf46c7c-8e02-4041-800b-126f241b1972 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 11:09:57.834715 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:09:57.834715 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:09:57.834715 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:09:57.834715 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:09:57.834715 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:09:57.834715 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:09:57.834715 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:09:57.834715 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:09:57.834715 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:09:57.834715 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:09:57.834715 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:09:57.834715 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 11:09:57.834715 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 11:09:57.834715 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 11:09:57.835817 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 11:09:57.835817 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:09:57.835817 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:09:57.835817 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:09:57.835817 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:09:57.835817 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:09:57.835817 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:09:57.835817 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:09:57.835817 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jun 29 11:09:57.835817 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:09:57.835817 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:09:57.835817 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:09:57.835817 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:09:57.835817 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:09:57.835817 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:09:57.835817 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:09:57.835817 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:09:57.837139 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 11:09:57.837139 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 11:09:57.837139 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 11:09:57.837139 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 11:09:57.837139 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 11:09:57.837139 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 11:09:57.837139 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 11:09:57.837139 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:09:57.837139 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 605/1217] 10.222.0.44 () {56 vars in 1111 bytes} [Wed Jun 29 11:09:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:09:57.841767 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d428a7a-c8d2-4682-84bd-8f3abd25a677 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:09:57.939916 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d428a7a-c8d2-4682-84bd-8f3abd25a677 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:09:57.962426 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:09:57.974038 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3221d81-0807-4a9f-a7a8-0eea28e871d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:09:57.974685 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3221d81-0807-4a9f-a7a8-0eea28e871d7 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:09:57.974965 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3221d81-0807-4a9f-a7a8-0eea28e871d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:09:57.975237 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3221d81-0807-4a9f-a7a8-0eea28e871d7 None admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:09:57.976034 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3221d81-0807-4a9f-a7a8-0eea28e871d7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:09:57.976383 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3221d81-0807-4a9f-a7a8-0eea28e871d7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:09:57.977796 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3221d81-0807-4a9f-a7a8-0eea28e871d7 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:09:57.989948 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 613/1218] 10.222.0.44 () {56 vars in 1148 bytes} [Wed Jun 29 11:09:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:09:57.999616 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf46c7c-8e02-4041-800b-126f241b1972 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:09:58.095793 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf46c7c-8e02-4041-800b-126f241b1972 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:09:58.123989 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:09:58.130279 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae13c23-c438-4828-bbf1-27e43a9e647f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:09:58.130926 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae13c23-c438-4828-bbf1-27e43a9e647f None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:09:58.131209 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae13c23-c438-4828-bbf1-27e43a9e647f None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:09:58.131485 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae13c23-c438-4828-bbf1-27e43a9e647f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:09:58.132339 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae13c23-c438-4828-bbf1-27e43a9e647f None admin] RBAC: Adding query filter params (role.id=a8212be3426741cbb1b38b1646c66405, scope.project.id=ebe5dffa4bb8440286687daefb6c98f4, user.id=1a0505342b704d02977a5125957c2c9e) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:09:58.132649 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae13c23-c438-4828-bbf1-27e43a9e647f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:09:58.134062 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae13c23-c438-4828-bbf1-27e43a9e647f None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:09:58.139284 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 606/1219] 10.222.0.44 () {56 vars in 1374 bytes} [Wed Jun 29 11:09:57 2022] GET /identity/v3/role_assignments?user.id=1a0505342b704d02977a5125957c2c9e&scope.project.id=ebe5dffa4bb8440286687daefb6c98f4&role.id=a8212be3426741cbb1b38b1646c66405 => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:10:00.157271 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e836e9-ca4c-4624-8be2-0543e8f2fad2 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:10:00.157818 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e836e9-ca4c-4624-8be2-0543e8f2fad2 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:10:00.157818 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e836e9-ca4c-4624-8be2-0543e8f2fad2 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:10:00.159213 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 614/1220] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:10:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:10:00.164450 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e040000-3cc5-490c-a108-bd5b1bfefbb1 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:10:00.164843 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e040000-3cc5-490c-a108-bd5b1bfefbb1 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:10:00.165002 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e040000-3cc5-490c-a108-bd5b1bfefbb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:10:00.182248 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-2e040000-3cc5-490c-a108-bd5b1bfefbb1 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:10:00.183490 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-2e040000-3cc5-490c-a108-bd5b1bfefbb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:10:00.249187 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 607/1221] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:10:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 11:10:00.273849 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e836e9-ca4c-4624-8be2-0543e8f2fad2 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:10:00.374309 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e836e9-ca4c-4624-8be2-0543e8f2fad2 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:10:00.407327 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc491213-be1b-48c8-946a-8674ea659147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:10:00.408139 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc491213-be1b-48c8-946a-8674ea659147 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:10:00.408423 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc491213-be1b-48c8-946a-8674ea659147 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:10:00.408694 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc491213-be1b-48c8-946a-8674ea659147 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:10:00.418628 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc491213-be1b-48c8-946a-8674ea659147 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 11:10:00.427091 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-fc491213-be1b-48c8-946a-8674ea659147 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 29 11:10:00.427091 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:10:00.427091 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:10:00.427091 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:10:00.427091 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:10:00.427091 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:10:00.427091 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:10:00.427091 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:10:00.427091 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:10:00.427091 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:10:00.427091 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:10:00.427091 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:10:00.427091 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 11:10:00.427091 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 11:10:00.427091 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 11:10:00.427091 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 11:10:00.428230 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 11:10:00.428230 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 11:10:00.428230 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 11:10:00.428230 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 11:10:00.428230 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:10:00.428230 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:10:00.428230 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:10:00.428230 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:10:00.428230 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:10:00.428230 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:10:00.428230 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:10:00.428230 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jun 29 11:10:00.428230 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:10:00.428230 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:10:00.428230 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:10:00.428230 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:10:00.429352 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:10:00.429352 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:10:00.429352 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:10:00.429352 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:10:00.429352 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 11:10:00.429352 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 11:10:00.429352 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 11:10:00.429352 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 11:10:00.429352 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 11:10:00.429352 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 11:10:00.429352 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 29 11:10:00.429352 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jun 29 11:10:00.429352 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 615/1222] 10.222.0.44 () {56 vars in 1102 bytes} [Wed Jun 29 11:10:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:10:00.434599 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e040000-3cc5-490c-a108-bd5b1bfefbb1 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:10:00.531318 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e040000-3cc5-490c-a108-bd5b1bfefbb1 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:10:00.553793 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:10:00.564127 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67812b9b-303a-4751-814b-9d04c4061f7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:10:00.564788 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67812b9b-303a-4751-814b-9d04c4061f7e None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:10:00.565072 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67812b9b-303a-4751-814b-9d04c4061f7e None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:10:00.565348 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67812b9b-303a-4751-814b-9d04c4061f7e None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:10:00.566598 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67812b9b-303a-4751-814b-9d04c4061f7e None admin] RBAC: Adding query filter params (name=service) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:10:00.566902 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67812b9b-303a-4751-814b-9d04c4061f7e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:10:00.568343 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67812b9b-303a-4751-814b-9d04c4061f7e None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:10:00.576385 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 608/1223] 10.222.0.44 () {56 vars in 1103 bytes} [Wed Jun 29 11:10:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:10:00.582912 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc491213-be1b-48c8-946a-8674ea659147 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:10:00.624805 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:10:00.680739 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc491213-be1b-48c8-946a-8674ea659147 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:10:00.713570 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5701bd07-a5b2-4b4c-be4e-3af15922ef99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:10:00.714352 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5701bd07-a5b2-4b4c-be4e-3af15922ef99 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:10:00.714636 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5701bd07-a5b2-4b4c-be4e-3af15922ef99 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:10:00.714908 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5701bd07-a5b2-4b4c-be4e-3af15922ef99 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:10:00.726123 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5701bd07-a5b2-4b4c-be4e-3af15922ef99 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:10:00.727551 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5701bd07-a5b2-4b4c-be4e-3af15922ef99 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:10:00.745203 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-5701bd07-a5b2-4b4c-be4e-3af15922ef99 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 11:10:00.745203 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:10:00.745203 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:10:00.745203 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:10:00.745203 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:10:00.745203 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:10:00.745203 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:10:00.745203 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:10:00.745203 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:10:00.745203 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:10:00.745203 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:10:00.745203 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:10:00.745203 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 11:10:00.745203 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 11:10:00.745203 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 11:10:00.746190 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 11:10:00.746190 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:10:00.746190 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:10:00.746190 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:10:00.746190 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:10:00.746190 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:10:00.746190 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:10:00.746190 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:10:00.746190 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jun 29 11:10:00.746190 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:10:00.746190 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:10:00.746190 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:10:00.746190 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:10:00.746190 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:10:00.746190 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:10:00.746190 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:10:00.746190 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:10:00.747170 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 11:10:00.747170 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 11:10:00.747170 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 11:10:00.747170 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jun 29 11:10:00.747170 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 616/1224] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jun 29 11:10:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:10:00.752452 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67812b9b-303a-4751-814b-9d04c4061f7e None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:10:00.852244 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67812b9b-303a-4751-814b-9d04c4061f7e None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:10:00.884578 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b097c2-7051-4f1b-8822-54d310dcc116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:10:00.885259 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b097c2-7051-4f1b-8822-54d310dcc116 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:10:00.885537 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b097c2-7051-4f1b-8822-54d310dcc116 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:10:00.885812 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b097c2-7051-4f1b-8822-54d310dcc116 None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:10:00.886515 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b097c2-7051-4f1b-8822-54d310dcc116 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:10:00.886816 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b097c2-7051-4f1b-8822-54d310dcc116 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:10:00.888183 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b097c2-7051-4f1b-8822-54d310dcc116 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:10:00.900295 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 609/1225] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Jun 29 11:10:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 11:10:00.906017 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5701bd07-a5b2-4b4c-be4e-3af15922ef99 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:10:00.996657 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:10:01.002994 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5701bd07-a5b2-4b4c-be4e-3af15922ef99 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:10:01.035199 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0846b933-425f-4941-912b-e282005cebe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:10:01.036014 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0846b933-425f-4941-912b-e282005cebe9 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:10:01.036296 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0846b933-425f-4941-912b-e282005cebe9 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:10:01.036570 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0846b933-425f-4941-912b-e282005cebe9 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:10:01.043765 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0846b933-425f-4941-912b-e282005cebe9 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:10:01.045217 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0846b933-425f-4941-912b-e282005cebe9 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:10:01.059123 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-0846b933-425f-4941-912b-e282005cebe9 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 29 11:10:01.059123 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:10:01.059123 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:10:01.059123 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:10:01.059123 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:10:01.059123 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:10:01.059123 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:10:01.059123 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:10:01.059123 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:10:01.059123 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:10:01.059123 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:10:01.059123 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:10:01.059123 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 11:10:01.059123 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 11:10:01.059123 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 11:10:01.059123 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 11:10:01.060303 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:10:01.060303 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:10:01.060303 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 11:10:01.060303 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 11:10:01.060303 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 11:10:01.060303 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 11:10:01.060303 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:10:01.060303 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:10:01.060303 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:10:01.060303 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:10:01.060303 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:10:01.060303 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jun 29 11:10:01.060303 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:10:01.060303 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:10:01.060303 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:10:01.060303 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:10:01.060303 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:10:01.060303 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:10:01.061484 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:10:01.061484 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:10:01.061484 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 11:10:01.061484 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 11:10:01.061484 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 11:10:01.061484 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 11:10:01.061484 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 11:10:01.061484 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 11:10:01.061484 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 29 11:10:01.061484 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jun 29 11:10:01.061484 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 617/1226] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jun 29 11:10:00 2022] GET /identity/v3/users/placement => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:10:01.066287 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b097c2-7051-4f1b-8822-54d310dcc116 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:10:01.112666 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:10:01.163218 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b097c2-7051-4f1b-8822-54d310dcc116 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:10:01.195235 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603da3de-371c-44e5-82b1-cea832153349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:10:01.196176 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603da3de-371c-44e5-82b1-cea832153349 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:10:01.196456 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603da3de-371c-44e5-82b1-cea832153349 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:10:01.196729 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603da3de-371c-44e5-82b1-cea832153349 None admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:10:01.197605 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603da3de-371c-44e5-82b1-cea832153349 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:10:01.197912 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603da3de-371c-44e5-82b1-cea832153349 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:10:01.199313 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603da3de-371c-44e5-82b1-cea832153349 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:10:01.216169 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 610/1227] 10.222.0.44 () {56 vars in 1143 bytes} [Wed Jun 29 11:10:01 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:10:01.221891 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0846b933-425f-4941-912b-e282005cebe9 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:10:01.321895 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0846b933-425f-4941-912b-e282005cebe9 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:10:01.357074 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87b8647-00c4-4716-a7cc-e2650f51b378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:10:01.357866 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87b8647-00c4-4716-a7cc-e2650f51b378 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:10:01.358149 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87b8647-00c4-4716-a7cc-e2650f51b378 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:10:01.358425 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87b8647-00c4-4716-a7cc-e2650f51b378 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:10:01.369160 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87b8647-00c4-4716-a7cc-e2650f51b378 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:10:01.370644 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87b8647-00c4-4716-a7cc-e2650f51b378 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:10:01.388075 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-f87b8647-00c4-4716-a7cc-e2650f51b378 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 11:10:01.388075 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:10:01.388075 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:10:01.388075 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:10:01.388075 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:10:01.388075 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:10:01.388075 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:10:01.388075 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:10:01.388075 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:10:01.388075 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:10:01.388075 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:10:01.388075 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:10:01.388075 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 11:10:01.388075 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 11:10:01.388075 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 11:10:01.389189 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 11:10:01.389189 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:10:01.389189 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:10:01.389189 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:10:01.389189 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:10:01.389189 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:10:01.389189 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:10:01.389189 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:10:01.389189 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jun 29 11:10:01.389189 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:10:01.389189 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:10:01.389189 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:10:01.389189 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:10:01.389189 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:10:01.389189 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:10:01.389189 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:10:01.389189 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:10:01.390323 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 11:10:01.390323 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 11:10:01.390323 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 11:10:01.390323 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jun 29 11:10:01.390323 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 618/1228] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jun 29 11:10:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:10:01.396050 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603da3de-371c-44e5-82b1-cea832153349 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:10:01.491940 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603da3de-371c-44e5-82b1-cea832153349 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:10:01.494183 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:10:01.525675 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d1b4a1-fdf5-4eab-9d0f-3f9ffeed0a13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:10:01.526341 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d1b4a1-fdf5-4eab-9d0f-3f9ffeed0a13 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:10:01.526622 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d1b4a1-fdf5-4eab-9d0f-3f9ffeed0a13 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:10:01.526896 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d1b4a1-fdf5-4eab-9d0f-3f9ffeed0a13 None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:10:01.527619 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d1b4a1-fdf5-4eab-9d0f-3f9ffeed0a13 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:10:01.527944 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d1b4a1-fdf5-4eab-9d0f-3f9ffeed0a13 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:10:01.529291 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d1b4a1-fdf5-4eab-9d0f-3f9ffeed0a13 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:10:01.540473 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 611/1229] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Jun 29 11:10:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:10:01.546114 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87b8647-00c4-4716-a7cc-e2650f51b378 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:10:01.555360 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:10:01.643249 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87b8647-00c4-4716-a7cc-e2650f51b378 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:10:01.675874 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0bf0f7-db90-4cb4-99f3-fc08fd43ca7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:10:01.676655 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0bf0f7-db90-4cb4-99f3-fc08fd43ca7b None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:10:01.676938 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0bf0f7-db90-4cb4-99f3-fc08fd43ca7b None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:10:01.677212 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0bf0f7-db90-4cb4-99f3-fc08fd43ca7b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:10:01.683668 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0bf0f7-db90-4cb4-99f3-fc08fd43ca7b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:10:01.687667 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0bf0f7-db90-4cb4-99f3-fc08fd43ca7b None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:10:01.699986 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-7a0bf0f7-db90-4cb4-99f3-fc08fd43ca7b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 11:10:01.699986 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:10:01.699986 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:10:01.699986 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:10:01.699986 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:10:01.699986 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:10:01.699986 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:10:01.699986 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:10:01.699986 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:10:01.699986 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:10:01.699986 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:10:01.699986 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:10:01.699986 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 11:10:01.699986 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 11:10:01.699986 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 11:10:01.700913 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 11:10:01.700913 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:10:01.700913 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:10:01.700913 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:10:01.700913 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:10:01.700913 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:10:01.700913 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:10:01.700913 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:10:01.700913 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jun 29 11:10:01.700913 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:10:01.700913 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:10:01.700913 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:10:01.700913 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:10:01.700913 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:10:01.700913 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:10:01.700913 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:10:01.700913 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:10:01.701857 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 11:10:01.701857 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 11:10:01.701857 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 11:10:01.701857 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 11:10:01.701857 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 11:10:01.701857 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 11:10:01.701857 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 11:10:01.701857 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jun 29 11:10:01.701857 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 619/1230] 10.222.0.44 () {56 vars in 1111 bytes} [Wed Jun 29 11:10:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:10:01.707616 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d1b4a1-fdf5-4eab-9d0f-3f9ffeed0a13 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:10:01.808175 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d1b4a1-fdf5-4eab-9d0f-3f9ffeed0a13 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:10:01.840939 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2250ef0f-77ec-4d64-8372-4333936bb543 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:10:01.841596 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2250ef0f-77ec-4d64-8372-4333936bb543 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:10:01.841881 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2250ef0f-77ec-4d64-8372-4333936bb543 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:10:01.842160 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2250ef0f-77ec-4d64-8372-4333936bb543 None admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:10:01.842940 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2250ef0f-77ec-4d64-8372-4333936bb543 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:10:01.843252 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2250ef0f-77ec-4d64-8372-4333936bb543 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:10:01.844704 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2250ef0f-77ec-4d64-8372-4333936bb543 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:10:01.857573 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 612/1231] 10.222.0.44 () {56 vars in 1148 bytes} [Wed Jun 29 11:10:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:10:01.863542 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0bf0f7-db90-4cb4-99f3-fc08fd43ca7b None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:10:01.927476 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:10:01.960707 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0bf0f7-db90-4cb4-99f3-fc08fd43ca7b None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:10:01.994428 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fafa9d-194d-4d4a-99c6-264ea7d9995e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:10:01.995290 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fafa9d-194d-4d4a-99c6-264ea7d9995e None admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:10:01.995569 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fafa9d-194d-4d4a-99c6-264ea7d9995e None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:10:01.995865 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fafa9d-194d-4d4a-99c6-264ea7d9995e None admin] PATH_INFO: `/v3/projects/ebe5dffa4bb8440286687daefb6c98f4/users/1a0505342b704d02977a5125957c2c9e/roles/a8212be3426741cbb1b38b1646c66405` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:10:02.010466 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fafa9d-194d-4d4a-99c6-264ea7d9995e None admin] RBAC: Authorizing `identity:create_grant(project_id=ebe5dffa4bb8440286687daefb6c98f4, user_id=1a0505342b704d02977a5125957c2c9e, role_id=a8212be3426741cbb1b38b1646c66405)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:10:02.011920 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fafa9d-194d-4d4a-99c6-264ea7d9995e None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:10:02.089257 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 620/1232] 10.222.0.44 () {58 vars in 1439 bytes} [Wed Jun 29 11:10:01 2022] PUT /identity/v3/projects/ebe5dffa4bb8440286687daefb6c98f4/users/1a0505342b704d02977a5125957c2c9e/roles/a8212be3426741cbb1b38b1646c66405 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 11:10:04.077074 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594b0c7e-2533-4acc-9248-127e5faf948e None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:10:04.077620 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594b0c7e-2533-4acc-9248-127e5faf948e None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:10:04.077731 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594b0c7e-2533-4acc-9248-127e5faf948e None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:10:04.079207 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 613/1233] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:10:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:10:04.084431 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195ea66f-3c53-4a4b-917b-c797bb8f3b3e None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:10:04.084867 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195ea66f-3c53-4a4b-917b-c797bb8f3b3e None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:10:04.084988 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195ea66f-3c53-4a4b-917b-c797bb8f3b3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:10:04.100740 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-195ea66f-3c53-4a4b-917b-c797bb8f3b3e None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:10:04.101950 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-195ea66f-3c53-4a4b-917b-c797bb8f3b3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:10:04.167978 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 621/1234] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:10:04 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 29 11:10:04.192117 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594b0c7e-2533-4acc-9248-127e5faf948e None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:10:04.216743 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:10:04.289771 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594b0c7e-2533-4acc-9248-127e5faf948e None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:10:04.323240 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad5df7b-7947-4143-8910-04b5ac8cf1d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:10:04.324059 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad5df7b-7947-4143-8910-04b5ac8cf1d1 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:10:04.324343 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad5df7b-7947-4143-8910-04b5ac8cf1d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:10:04.324613 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad5df7b-7947-4143-8910-04b5ac8cf1d1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:10:04.334720 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad5df7b-7947-4143-8910-04b5ac8cf1d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 11:10:04.342411 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-5ad5df7b-7947-4143-8910-04b5ac8cf1d1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 29 11:10:04.342411 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:10:04.342411 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:10:04.342411 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:10:04.342411 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:10:04.342411 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:10:04.342411 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:10:04.342411 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:10:04.342411 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:10:04.342411 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:10:04.342411 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:10:04.342411 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:10:04.342411 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 11:10:04.342411 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 11:10:04.342411 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 11:10:04.342411 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 11:10:04.343504 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 11:10:04.343504 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 11:10:04.343504 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 11:10:04.343504 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 11:10:04.343504 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:10:04.343504 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:10:04.343504 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:10:04.343504 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:10:04.343504 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:10:04.343504 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:10:04.343504 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:10:04.343504 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jun 29 11:10:04.343504 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:10:04.343504 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:10:04.343504 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:10:04.343504 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:10:04.344511 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:10:04.344511 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:10:04.344511 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:10:04.344511 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:10:04.344511 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 11:10:04.344511 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 11:10:04.344511 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 11:10:04.344511 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 11:10:04.344511 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 11:10:04.344511 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 11:10:04.344511 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 29 11:10:04.344511 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:10:04.344511 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 614/1235] 10.222.0.44 () {56 vars in 1102 bytes} [Wed Jun 29 11:10:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:10:04.350206 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195ea66f-3c53-4a4b-917b-c797bb8f3b3e None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:10:04.396653 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:10:04.448414 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195ea66f-3c53-4a4b-917b-c797bb8f3b3e None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:10:04.481221 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc38484f-7f3d-4940-a3c6-b2c4da8e3a69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:10:04.481886 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc38484f-7f3d-4940-a3c6-b2c4da8e3a69 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:10:04.482167 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc38484f-7f3d-4940-a3c6-b2c4da8e3a69 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:10:04.482439 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc38484f-7f3d-4940-a3c6-b2c4da8e3a69 None admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:10:04.483181 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc38484f-7f3d-4940-a3c6-b2c4da8e3a69 None admin] RBAC: Adding query filter params (name=service) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:10:04.483487 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc38484f-7f3d-4940-a3c6-b2c4da8e3a69 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:10:04.484898 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc38484f-7f3d-4940-a3c6-b2c4da8e3a69 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:10:04.493545 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 622/1236] 10.222.0.44 () {56 vars in 1103 bytes} [Wed Jun 29 11:10:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:10:04.499373 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad5df7b-7947-4143-8910-04b5ac8cf1d1 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:10:04.581681 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:10:04.597327 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad5df7b-7947-4143-8910-04b5ac8cf1d1 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:10:04.630471 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d3ec31-5d8a-45e7-9d7d-219926e6e4d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:10:04.631256 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d3ec31-5d8a-45e7-9d7d-219926e6e4d1 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:10:04.631540 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d3ec31-5d8a-45e7-9d7d-219926e6e4d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:10:04.631817 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d3ec31-5d8a-45e7-9d7d-219926e6e4d1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:10:04.641628 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d3ec31-5d8a-45e7-9d7d-219926e6e4d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:10:04.643072 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d3ec31-5d8a-45e7-9d7d-219926e6e4d1 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:10:04.661975 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-35d3ec31-5d8a-45e7-9d7d-219926e6e4d1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 11:10:04.661975 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:10:04.661975 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:10:04.661975 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:10:04.661975 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:10:04.661975 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:10:04.661975 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:10:04.661975 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:10:04.661975 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:10:04.661975 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:10:04.661975 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:10:04.661975 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:10:04.661975 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 11:10:04.661975 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 11:10:04.661975 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 11:10:04.663008 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 11:10:04.663008 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:10:04.663008 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:10:04.663008 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:10:04.663008 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:10:04.663008 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:10:04.663008 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:10:04.663008 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:10:04.663008 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jun 29 11:10:04.663008 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:10:04.663008 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:10:04.663008 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:10:04.663008 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:10:04.663008 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:10:04.663008 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:10:04.663008 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:10:04.663008 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:10:04.664280 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 11:10:04.664280 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 11:10:04.664280 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 11:10:04.664280 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:10:04.664280 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 615/1237] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jun 29 11:10:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:10:04.668959 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc38484f-7f3d-4940-a3c6-b2c4da8e3a69 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:10:04.764450 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc38484f-7f3d-4940-a3c6-b2c4da8e3a69 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:10:04.773887 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:10:04.797878 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da1159c-d219-424e-8f9a-3e22033acc20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:10:04.798534 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da1159c-d219-424e-8f9a-3e22033acc20 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:10:04.798819 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da1159c-d219-424e-8f9a-3e22033acc20 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:10:04.799097 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da1159c-d219-424e-8f9a-3e22033acc20 None admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:10:04.799809 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da1159c-d219-424e-8f9a-3e22033acc20 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:10:04.800188 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da1159c-d219-424e-8f9a-3e22033acc20 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:10:04.801546 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da1159c-d219-424e-8f9a-3e22033acc20 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:10:04.812365 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 623/1238] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Jun 29 11:10:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:10:04.818062 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d3ec31-5d8a-45e7-9d7d-219926e6e4d1 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:10:04.920129 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d3ec31-5d8a-45e7-9d7d-219926e6e4d1 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:10:04.953004 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e542f3ae-0f76-444f-90e0-ea4c37cb483d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:10:04.953792 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e542f3ae-0f76-444f-90e0-ea4c37cb483d None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:10:04.954068 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e542f3ae-0f76-444f-90e0-ea4c37cb483d None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:10:04.954341 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e542f3ae-0f76-444f-90e0-ea4c37cb483d None admin] PATH_INFO: `/v3/users/placement` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:10:04.957514 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:10:04.961894 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e542f3ae-0f76-444f-90e0-ea4c37cb483d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:10:04.963317 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e542f3ae-0f76-444f-90e0-ea4c37cb483d None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:10:04.978136 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-e542f3ae-0f76-444f-90e0-ea4c37cb483d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 29 11:10:04.978136 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:10:04.978136 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:10:04.978136 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:10:04.978136 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:10:04.978136 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:10:04.978136 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:10:04.978136 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:10:04.978136 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:10:04.978136 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:10:04.978136 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:10:04.978136 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:10:04.978136 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 11:10:04.978136 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 11:10:04.978136 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 11:10:04.978136 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 11:10:04.979927 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:10:04.979927 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:10:04.979927 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 11:10:04.979927 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 11:10:04.979927 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 11:10:04.979927 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 11:10:04.979927 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:10:04.979927 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:10:04.979927 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:10:04.979927 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:10:04.979927 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:10:04.979927 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jun 29 11:10:04.979927 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:10:04.979927 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:10:04.979927 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:10:04.979927 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:10:04.979927 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:10:04.979927 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:10:04.982094 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:10:04.982094 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:10:04.982094 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 11:10:04.982094 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 11:10:04.982094 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 11:10:04.982094 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 11:10:04.982094 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 11:10:04.982094 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 11:10:04.982094 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 29 11:10:04.982094 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:10:04.982094 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 616/1239] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jun 29 11:10:04 2022] GET /identity/v3/users/placement => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:10:04.986716 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da1159c-d219-424e-8f9a-3e22033acc20 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:10:05.087472 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da1159c-d219-424e-8f9a-3e22033acc20 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:10:05.119791 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd5f84a-5cae-455a-b945-7947c916698f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:10:05.120484 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd5f84a-5cae-455a-b945-7947c916698f None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:10:05.120772 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd5f84a-5cae-455a-b945-7947c916698f None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:10:05.121047 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd5f84a-5cae-455a-b945-7947c916698f None admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:10:05.121878 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd5f84a-5cae-455a-b945-7947c916698f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:10:05.122189 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd5f84a-5cae-455a-b945-7947c916698f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:10:05.123667 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd5f84a-5cae-455a-b945-7947c916698f None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:10:05.140993 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 624/1240] 10.222.0.44 () {56 vars in 1143 bytes} [Wed Jun 29 11:10:04 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:10:05.147456 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e542f3ae-0f76-444f-90e0-ea4c37cb483d None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:10:05.243145 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e542f3ae-0f76-444f-90e0-ea4c37cb483d None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:10:05.276536 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca5ef65-ee3f-4bba-ae36-c575c0eda63b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:10:05.277171 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca5ef65-ee3f-4bba-ae36-c575c0eda63b None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:10:05.277451 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca5ef65-ee3f-4bba-ae36-c575c0eda63b None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:10:05.277722 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca5ef65-ee3f-4bba-ae36-c575c0eda63b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:10:05.337158 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:10:05.343685 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca5ef65-ee3f-4bba-ae36-c575c0eda63b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:10:05.345095 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca5ef65-ee3f-4bba-ae36-c575c0eda63b None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:10:05.443383 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 617/1241] 10.222.0.44 () {58 vars in 1282 bytes} [Wed Jun 29 11:10:05 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 29 11:10:05.449441 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd5f84a-5cae-455a-b945-7947c916698f None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:10:05.477404 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:10:05.545235 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd5f84a-5cae-455a-b945-7947c916698f None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:10:05.579173 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7290de0-9a34-4070-be31-d92ca08924b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:10:05.579980 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7290de0-9a34-4070-be31-d92ca08924b2 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:10:05.580262 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7290de0-9a34-4070-be31-d92ca08924b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:10:05.580538 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7290de0-9a34-4070-be31-d92ca08924b2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:10:05.590746 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7290de0-9a34-4070-be31-d92ca08924b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:10:05.594179 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7290de0-9a34-4070-be31-d92ca08924b2 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:10:05.608970 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-f7290de0-9a34-4070-be31-d92ca08924b2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 11:10:05.608970 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:10:05.608970 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:10:05.608970 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:10:05.608970 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:10:05.608970 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:10:05.608970 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:10:05.608970 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:10:05.608970 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:10:05.608970 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:10:05.608970 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:10:05.608970 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:10:05.608970 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 11:10:05.608970 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 11:10:05.608970 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 11:10:05.610004 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 11:10:05.610004 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:10:05.610004 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:10:05.610004 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:10:05.610004 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:10:05.610004 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:10:05.610004 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:10:05.610004 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:10:05.610004 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jun 29 11:10:05.610004 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:10:05.610004 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:10:05.610004 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:10:05.610004 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:10:05.610004 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:10:05.610004 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:10:05.610004 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:10:05.610004 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:10:05.611021 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 11:10:05.611021 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 11:10:05.611021 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 11:10:05.611021 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jun 29 11:10:05.611021 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 625/1242] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jun 29 11:10:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:10:05.616448 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca5ef65-ee3f-4bba-ae36-c575c0eda63b None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:10:05.706612 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:10:05.712181 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca5ef65-ee3f-4bba-ae36-c575c0eda63b None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:10:05.744573 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae9b9a1-6c61-4e52-847b-f04ab20adac9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:10:05.745211 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae9b9a1-6c61-4e52-847b-f04ab20adac9 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:10:05.745493 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae9b9a1-6c61-4e52-847b-f04ab20adac9 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:10:05.745769 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae9b9a1-6c61-4e52-847b-f04ab20adac9 None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:10:05.746489 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae9b9a1-6c61-4e52-847b-f04ab20adac9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:10:05.746797 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae9b9a1-6c61-4e52-847b-f04ab20adac9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:10:05.748187 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae9b9a1-6c61-4e52-847b-f04ab20adac9 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:10:05.760467 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 618/1243] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Jun 29 11:10:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 11:10:05.766383 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7290de0-9a34-4070-be31-d92ca08924b2 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:10:05.841817 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:10:05.862482 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7290de0-9a34-4070-be31-d92ca08924b2 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:10:05.894504 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0c5b56-2bdb-4c25-bdb2-b1e0fd972059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:10:05.895278 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0c5b56-2bdb-4c25-bdb2-b1e0fd972059 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:10:05.895559 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0c5b56-2bdb-4c25-bdb2-b1e0fd972059 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:10:05.895858 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0c5b56-2bdb-4c25-bdb2-b1e0fd972059 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:10:05.902235 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0c5b56-2bdb-4c25-bdb2-b1e0fd972059 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:10:05.903658 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0c5b56-2bdb-4c25-bdb2-b1e0fd972059 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:10:05.917511 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-4e0c5b56-2bdb-4c25-bdb2-b1e0fd972059 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 11:10:05.917511 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:10:05.917511 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:10:05.917511 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:10:05.917511 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:10:05.917511 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:10:05.917511 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:10:05.917511 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:10:05.917511 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:10:05.917511 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:10:05.917511 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:10:05.917511 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:10:05.917511 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 11:10:05.917511 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 11:10:05.917511 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 11:10:05.918582 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 11:10:05.918582 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:10:05.918582 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:10:05.918582 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:10:05.918582 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:10:05.918582 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:10:05.918582 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:10:05.918582 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:10:05.918582 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jun 29 11:10:05.918582 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:10:05.918582 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:10:05.918582 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:10:05.918582 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:10:05.918582 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:10:05.918582 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:10:05.918582 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:10:05.918582 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:10:05.919685 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 11:10:05.919685 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 11:10:05.919685 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 11:10:05.919685 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 11:10:05.919685 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 11:10:05.919685 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 11:10:05.919685 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 11:10:05.919685 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jun 29 11:10:05.919685 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 626/1244] 10.222.0.44 () {56 vars in 1111 bytes} [Wed Jun 29 11:10:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:10:05.924449 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae9b9a1-6c61-4e52-847b-f04ab20adac9 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:10:06.022223 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae9b9a1-6c61-4e52-847b-f04ab20adac9 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:10:06.054691 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc2fcf2-893b-4d8f-a1df-211396b21a8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:10:06.055329 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc2fcf2-893b-4d8f-a1df-211396b21a8e None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:10:06.055609 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc2fcf2-893b-4d8f-a1df-211396b21a8e None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:10:06.055952 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc2fcf2-893b-4d8f-a1df-211396b21a8e None admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:10:06.056738 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc2fcf2-893b-4d8f-a1df-211396b21a8e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:10:06.057098 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc2fcf2-893b-4d8f-a1df-211396b21a8e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:10:06.058509 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc2fcf2-893b-4d8f-a1df-211396b21a8e None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:10:06.070341 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 619/1245] 10.222.0.44 () {56 vars in 1148 bytes} [Wed Jun 29 11:10:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:10:06.076680 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0c5b56-2bdb-4c25-bdb2-b1e0fd972059 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:10:06.173906 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0c5b56-2bdb-4c25-bdb2-b1e0fd972059 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:10:06.205816 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcd7bab-f659-4a6b-ad1a-aa1f6f0a2fd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:10:06.212563 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcd7bab-f659-4a6b-ad1a-aa1f6f0a2fd6 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:10:06.212563 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcd7bab-f659-4a6b-ad1a-aa1f6f0a2fd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:10:06.212563 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcd7bab-f659-4a6b-ad1a-aa1f6f0a2fd6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:10:06.212563 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcd7bab-f659-4a6b-ad1a-aa1f6f0a2fd6 None admin] RBAC: Adding query filter params (role.id=a8212be3426741cbb1b38b1646c66405, scope.project.id=ebe5dffa4bb8440286687daefb6c98f4, user.id=1a0505342b704d02977a5125957c2c9e) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:10:06.212563 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcd7bab-f659-4a6b-ad1a-aa1f6f0a2fd6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:10:06.212563 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcd7bab-f659-4a6b-ad1a-aa1f6f0a2fd6 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:10:06.215794 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 627/1246] 10.222.0.44 () {56 vars in 1374 bytes} [Wed Jun 29 11:10:06 2022] GET /identity/v3/role_assignments?user.id=1a0505342b704d02977a5125957c2c9e&scope.project.id=ebe5dffa4bb8440286687daefb6c98f4&role.id=a8212be3426741cbb1b38b1646c66405 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:10:08.417198 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa67b6c-f666-4aac-bde5-13a7738a5c16 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:10:08.417754 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa67b6c-f666-4aac-bde5-13a7738a5c16 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:10:08.417754 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa67b6c-f666-4aac-bde5-13a7738a5c16 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:10:08.419173 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 620/1247] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:10:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:10:08.424340 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff68aca-c945-46b1-b12f-5619b4d0eedf None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:10:08.424723 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff68aca-c945-46b1-b12f-5619b4d0eedf None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:10:08.424906 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff68aca-c945-46b1-b12f-5619b4d0eedf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:10:08.430040 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:10:08.446261 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-4ff68aca-c945-46b1-b12f-5619b4d0eedf None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:10:08.449388 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-4ff68aca-c945-46b1-b12f-5619b4d0eedf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:10:08.515205 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 628/1248] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:10:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 11:10:08.539407 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa67b6c-f666-4aac-bde5-13a7738a5c16 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:10:08.557102 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:10:08.637408 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa67b6c-f666-4aac-bde5-13a7738a5c16 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:10:08.672273 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec35fdb-940b-40a7-b85a-a85299bd8d00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:10:08.672989 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec35fdb-940b-40a7-b85a-a85299bd8d00 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:10:08.673269 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec35fdb-940b-40a7-b85a-a85299bd8d00 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:10:08.673539 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec35fdb-940b-40a7-b85a-a85299bd8d00 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:10:08.683424 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec35fdb-940b-40a7-b85a-a85299bd8d00 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 11:10:08.691155 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-4ec35fdb-940b-40a7-b85a-a85299bd8d00 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 11:10:08.691155 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:10:08.691155 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:10:08.691155 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:10:08.691155 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:10:08.691155 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:10:08.691155 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:10:08.691155 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:10:08.691155 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:10:08.691155 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:10:08.691155 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:10:08.691155 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:10:08.691155 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 11:10:08.691155 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 11:10:08.691155 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 11:10:08.691155 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 11:10:08.692318 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 11:10:08.692318 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 11:10:08.692318 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 11:10:08.692318 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 11:10:08.692318 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:10:08.692318 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:10:08.692318 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:10:08.692318 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:10:08.692318 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:10:08.692318 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:10:08.692318 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:10:08.692318 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jun 29 11:10:08.692318 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:10:08.692318 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:10:08.692318 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:10:08.692318 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:10:08.693570 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:10:08.693570 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:10:08.693570 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:10:08.693570 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:10:08.693570 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 11:10:08.693570 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 11:10:08.693570 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 11:10:08.693570 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 11:10:08.693570 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 11:10:08.693570 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 11:10:08.693570 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 11:10:08.693570 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:10:08.693570 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 621/1249] 10.222.0.44 () {56 vars in 1096 bytes} [Wed Jun 29 11:10:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:10:08.698574 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff68aca-c945-46b1-b12f-5619b4d0eedf None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:10:08.795546 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff68aca-c945-46b1-b12f-5619b4d0eedf None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:10:08.825345 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:10:08.830043 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37e2c3b-60a3-4ccb-9fd6-65892be3245b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:10:08.830710 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37e2c3b-60a3-4ccb-9fd6-65892be3245b None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:10:08.830995 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37e2c3b-60a3-4ccb-9fd6-65892be3245b None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:10:08.831272 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37e2c3b-60a3-4ccb-9fd6-65892be3245b None admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:10:08.832696 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37e2c3b-60a3-4ccb-9fd6-65892be3245b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:10:08.833006 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37e2c3b-60a3-4ccb-9fd6-65892be3245b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:10:08.834370 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37e2c3b-60a3-4ccb-9fd6-65892be3245b None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:10:08.842573 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 629/1250] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Jun 29 11:10:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:10:08.848248 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec35fdb-940b-40a7-b85a-a85299bd8d00 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:10:08.926272 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:10:08.948735 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec35fdb-940b-40a7-b85a-a85299bd8d00 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:10:08.980862 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f06f95d-4000-4db3-b7f7-525251600ddb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:10:08.981643 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f06f95d-4000-4db3-b7f7-525251600ddb None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:10:08.981927 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f06f95d-4000-4db3-b7f7-525251600ddb None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:10:08.982204 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f06f95d-4000-4db3-b7f7-525251600ddb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:10:08.992492 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f06f95d-4000-4db3-b7f7-525251600ddb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:10:08.996766 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f06f95d-4000-4db3-b7f7-525251600ddb None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:10:09.010841 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-1f06f95d-4000-4db3-b7f7-525251600ddb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 11:10:09.010841 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:10:09.010841 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:10:09.010841 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:10:09.010841 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:10:09.010841 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:10:09.010841 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:10:09.010841 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:10:09.010841 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:10:09.010841 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:10:09.010841 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:10:09.010841 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:10:09.010841 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 11:10:09.010841 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 11:10:09.010841 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 11:10:09.012022 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 11:10:09.012022 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:10:09.012022 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:10:09.012022 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:10:09.012022 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:10:09.012022 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:10:09.012022 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:10:09.012022 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:10:09.012022 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jun 29 11:10:09.012022 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:10:09.012022 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:10:09.012022 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:10:09.012022 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:10:09.012022 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:10:09.012022 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:10:09.012022 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:10:09.012022 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:10:09.013136 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 11:10:09.013136 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 11:10:09.013136 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 11:10:09.013136 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:10:09.013136 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 622/1251] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jun 29 11:10:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:10:09.020675 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37e2c3b-60a3-4ccb-9fd6-65892be3245b None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:10:09.123571 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37e2c3b-60a3-4ccb-9fd6-65892be3245b None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:10:09.156873 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbd5b48-a524-4c2d-a820-6aedea075404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:10:09.157532 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbd5b48-a524-4c2d-a820-6aedea075404 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:10:09.157812 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbd5b48-a524-4c2d-a820-6aedea075404 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:10:09.158086 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbd5b48-a524-4c2d-a820-6aedea075404 None admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:10:09.158848 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbd5b48-a524-4c2d-a820-6aedea075404 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:10:09.159152 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbd5b48-a524-4c2d-a820-6aedea075404 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:10:09.160549 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbd5b48-a524-4c2d-a820-6aedea075404 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:10:09.172981 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 630/1252] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Jun 29 11:10:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:10:09.179284 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f06f95d-4000-4db3-b7f7-525251600ddb None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:10:09.277620 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f06f95d-4000-4db3-b7f7-525251600ddb None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:10:09.302850 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:10:09.310332 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0362eb55-a271-48be-837a-17c850433cd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:10:09.311120 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0362eb55-a271-48be-837a-17c850433cd8 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:10:09.311401 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0362eb55-a271-48be-837a-17c850433cd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:10:09.311678 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0362eb55-a271-48be-837a-17c850433cd8 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:10:09.318901 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0362eb55-a271-48be-837a-17c850433cd8 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:10:09.320360 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0362eb55-a271-48be-837a-17c850433cd8 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:10:09.335064 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-0362eb55-a271-48be-837a-17c850433cd8 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 29 11:10:09.335064 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:10:09.335064 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:10:09.335064 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:10:09.335064 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:10:09.335064 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:10:09.335064 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:10:09.335064 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:10:09.335064 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:10:09.335064 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:10:09.335064 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:10:09.335064 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:10:09.335064 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 11:10:09.335064 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 11:10:09.335064 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 11:10:09.335064 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 11:10:09.336209 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:10:09.336209 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:10:09.336209 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 11:10:09.336209 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 11:10:09.336209 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 11:10:09.336209 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 11:10:09.336209 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:10:09.336209 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:10:09.336209 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:10:09.336209 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:10:09.336209 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:10:09.336209 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jun 29 11:10:09.336209 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:10:09.336209 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:10:09.336209 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:10:09.336209 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:10:09.336209 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:10:09.336209 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:10:09.337243 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:10:09.337243 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:10:09.337243 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 11:10:09.337243 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 11:10:09.337243 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 11:10:09.337243 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 11:10:09.337243 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 11:10:09.337243 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 11:10:09.337243 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 29 11:10:09.337243 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:10:09.337243 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 623/1253] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jun 29 11:10:09 2022] GET /identity/v3/users/placement => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:10:09.342070 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbd5b48-a524-4c2d-a820-6aedea075404 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:10:09.396421 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:10:09.439804 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbd5b48-a524-4c2d-a820-6aedea075404 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:10:09.473503 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3789e477-36a3-4501-9a18-5e32e912da2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:10:09.474164 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3789e477-36a3-4501-9a18-5e32e912da2f None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:10:09.474442 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3789e477-36a3-4501-9a18-5e32e912da2f None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:10:09.474712 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3789e477-36a3-4501-9a18-5e32e912da2f None admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:10:09.475658 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3789e477-36a3-4501-9a18-5e32e912da2f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:10:09.476008 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3789e477-36a3-4501-9a18-5e32e912da2f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:10:09.477516 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3789e477-36a3-4501-9a18-5e32e912da2f None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:10:09.494367 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 631/1254] 10.222.0.44 () {56 vars in 1143 bytes} [Wed Jun 29 11:10:09 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:10:09.500118 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0362eb55-a271-48be-837a-17c850433cd8 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:10:09.596188 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0362eb55-a271-48be-837a-17c850433cd8 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:10:09.628187 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72e0c83-e5b6-45c0-8721-427455f2ab0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:10:09.629333 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72e0c83-e5b6-45c0-8721-427455f2ab0f None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:10:09.629613 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72e0c83-e5b6-45c0-8721-427455f2ab0f None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:10:09.629891 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72e0c83-e5b6-45c0-8721-427455f2ab0f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:10:09.682670 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:10:09.694950 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72e0c83-e5b6-45c0-8721-427455f2ab0f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:10:09.696615 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72e0c83-e5b6-45c0-8721-427455f2ab0f None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:10:09.791288 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 624/1255] 10.222.0.44 () {58 vars in 1282 bytes} [Wed Jun 29 11:10:09 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 29 11:10:09.797054 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3789e477-36a3-4501-9a18-5e32e912da2f None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:10:09.895344 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3789e477-36a3-4501-9a18-5e32e912da2f None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:10:09.907996 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:10:09.930124 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad74c85-c457-4670-bde2-72484965c0cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:10:09.930919 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad74c85-c457-4670-bde2-72484965c0cd None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:10:09.931203 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad74c85-c457-4670-bde2-72484965c0cd None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:10:09.931477 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad74c85-c457-4670-bde2-72484965c0cd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:10:09.941997 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad74c85-c457-4670-bde2-72484965c0cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:10:09.943446 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad74c85-c457-4670-bde2-72484965c0cd None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:10:09.960770 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-9ad74c85-c457-4670-bde2-72484965c0cd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 11:10:09.960770 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:10:09.960770 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:10:09.960770 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:10:09.960770 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:10:09.960770 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:10:09.960770 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:10:09.960770 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:10:09.960770 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:10:09.960770 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:10:09.960770 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:10:09.960770 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:10:09.960770 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 11:10:09.960770 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 11:10:09.960770 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 11:10:09.961887 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 11:10:09.961887 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:10:09.961887 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:10:09.961887 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:10:09.961887 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:10:09.961887 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:10:09.961887 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:10:09.961887 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:10:09.961887 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jun 29 11:10:09.961887 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:10:09.961887 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:10:09.961887 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:10:09.961887 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:10:09.961887 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:10:09.961887 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:10:09.961887 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:10:09.961887 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:10:09.962828 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 11:10:09.962828 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 11:10:09.962828 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 11:10:09.962828 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jun 29 11:10:09.962828 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 632/1256] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jun 29 11:10:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:10:09.967721 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72e0c83-e5b6-45c0-8721-427455f2ab0f None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:10:10.053519 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:10:10.064898 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72e0c83-e5b6-45c0-8721-427455f2ab0f None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:10:10.097727 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4bb97d-a6c2-4646-9626-868cc5e022b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:10:10.099349 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4bb97d-a6c2-4646-9626-868cc5e022b5 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:10:10.099349 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4bb97d-a6c2-4646-9626-868cc5e022b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:10:10.099349 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4bb97d-a6c2-4646-9626-868cc5e022b5 None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:10:10.099674 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4bb97d-a6c2-4646-9626-868cc5e022b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:10:10.100011 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4bb97d-a6c2-4646-9626-868cc5e022b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:10:10.101359 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4bb97d-a6c2-4646-9626-868cc5e022b5 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:10:10.112272 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 625/1257] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Jun 29 11:10:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:10:10.120634 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad74c85-c457-4670-bde2-72484965c0cd None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:10:10.223221 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad74c85-c457-4670-bde2-72484965c0cd None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:10:10.255789 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e58b375-2458-4a97-8559-90acb6ed5cbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:10:10.256634 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e58b375-2458-4a97-8559-90acb6ed5cbd None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:10:10.256902 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e58b375-2458-4a97-8559-90acb6ed5cbd None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:10:10.257175 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e58b375-2458-4a97-8559-90acb6ed5cbd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:10:10.263678 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e58b375-2458-4a97-8559-90acb6ed5cbd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:10:10.265142 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e58b375-2458-4a97-8559-90acb6ed5cbd None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:10:10.279217 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-3e58b375-2458-4a97-8559-90acb6ed5cbd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 11:10:10.279217 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:10:10.279217 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:10:10.279217 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:10:10.279217 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:10:10.279217 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:10:10.279217 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:10:10.279217 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:10:10.279217 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:10:10.279217 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:10:10.279217 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:10:10.279217 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:10:10.279217 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 11:10:10.279217 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 11:10:10.279217 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 11:10:10.280274 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 11:10:10.280274 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:10:10.280274 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:10:10.280274 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:10:10.280274 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:10:10.280274 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:10:10.280274 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:10:10.280274 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:10:10.280274 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jun 29 11:10:10.280274 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:10:10.280274 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:10:10.280274 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:10:10.280274 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:10:10.280274 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:10:10.280274 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:10:10.280274 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:10:10.280274 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:10:10.281193 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 11:10:10.281193 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 11:10:10.281193 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 11:10:10.281193 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 11:10:10.281193 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 11:10:10.281193 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 11:10:10.281193 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 11:10:10.281193 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jun 29 11:10:10.281193 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 633/1258] 10.222.0.44 () {56 vars in 1111 bytes} [Wed Jun 29 11:10:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:10:10.286405 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4bb97d-a6c2-4646-9626-868cc5e022b5 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:10:10.384240 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4bb97d-a6c2-4646-9626-868cc5e022b5 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:10:10.417542 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7612de3a-e75c-41db-a4c1-520df9a6ee79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:10:10.418195 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7612de3a-e75c-41db-a4c1-520df9a6ee79 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:10:10.418475 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7612de3a-e75c-41db-a4c1-520df9a6ee79 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:10:10.418749 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7612de3a-e75c-41db-a4c1-520df9a6ee79 None admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:10:10.419520 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7612de3a-e75c-41db-a4c1-520df9a6ee79 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:10:10.419852 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7612de3a-e75c-41db-a4c1-520df9a6ee79 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:10:10.421301 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7612de3a-e75c-41db-a4c1-520df9a6ee79 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:10:10.433055 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 626/1259] 10.222.0.44 () {56 vars in 1148 bytes} [Wed Jun 29 11:10:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:10:10.439231 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e58b375-2458-4a97-8559-90acb6ed5cbd None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:10:10.472001 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:10:10.547511 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e58b375-2458-4a97-8559-90acb6ed5cbd None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:10:10.579500 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b2ba03-2228-4779-9b7b-2bdf690c9fa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:10:10.580166 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b2ba03-2228-4779-9b7b-2bdf690c9fa5 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:10:10.580447 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b2ba03-2228-4779-9b7b-2bdf690c9fa5 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:10:10.580722 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b2ba03-2228-4779-9b7b-2bdf690c9fa5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:10:10.581534 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b2ba03-2228-4779-9b7b-2bdf690c9fa5 None admin] RBAC: Adding query filter params (role.id=e95c2ce63e114cc19a9dda6ad21e794f, scope.project.id=ebe5dffa4bb8440286687daefb6c98f4, user.id=1a0505342b704d02977a5125957c2c9e) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:10:10.582467 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b2ba03-2228-4779-9b7b-2bdf690c9fa5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:10:10.583906 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b2ba03-2228-4779-9b7b-2bdf690c9fa5 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:10:10.589190 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 634/1260] 10.222.0.44 () {56 vars in 1374 bytes} [Wed Jun 29 11:10:10 2022] GET /identity/v3/role_assignments?user.id=1a0505342b704d02977a5125957c2c9e&scope.project.id=ebe5dffa4bb8440286687daefb6c98f4&role.id=e95c2ce63e114cc19a9dda6ad21e794f => generated 259 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:10:12.555966 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49808d45-358c-49e9-aee0-c58e1ec5cff4 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:10:12.556524 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49808d45-358c-49e9-aee0-c58e1ec5cff4 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:10:12.556634 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49808d45-358c-49e9-aee0-c58e1ec5cff4 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:10:12.557931 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 627/1261] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:10:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:10:12.563222 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd056d25-f2b2-49a3-b46f-f3d90b80b63b None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:10:12.563640 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd056d25-f2b2-49a3-b46f-f3d90b80b63b None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:10:12.563775 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd056d25-f2b2-49a3-b46f-f3d90b80b63b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:10:12.580672 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-cd056d25-f2b2-49a3-b46f-f3d90b80b63b None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:10:12.581861 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-cd056d25-f2b2-49a3-b46f-f3d90b80b63b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:10:12.647049 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 635/1262] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:10:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 11:10:12.669888 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49808d45-358c-49e9-aee0-c58e1ec5cff4 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:10:12.681210 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:10:12.768733 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49808d45-358c-49e9-aee0-c58e1ec5cff4 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:10:12.803405 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b876778-85e6-4b2a-b3ad-8473ea94732d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:10:12.804223 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b876778-85e6-4b2a-b3ad-8473ea94732d None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:10:12.804505 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b876778-85e6-4b2a-b3ad-8473ea94732d None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:10:12.804778 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b876778-85e6-4b2a-b3ad-8473ea94732d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:10:12.818949 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b876778-85e6-4b2a-b3ad-8473ea94732d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 11:10:12.825687 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-6b876778-85e6-4b2a-b3ad-8473ea94732d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 11:10:12.825687 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:10:12.825687 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:10:12.825687 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:10:12.825687 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:10:12.825687 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:10:12.825687 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:10:12.825687 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:10:12.825687 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:10:12.825687 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:10:12.825687 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:10:12.825687 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:10:12.825687 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 11:10:12.825687 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 11:10:12.825687 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 11:10:12.825687 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 11:10:12.826798 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 11:10:12.826798 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 11:10:12.826798 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 11:10:12.826798 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 11:10:12.826798 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:10:12.826798 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:10:12.826798 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:10:12.826798 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:10:12.826798 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:10:12.826798 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:10:12.826798 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:10:12.826798 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jun 29 11:10:12.826798 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:10:12.826798 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:10:12.826798 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:10:12.826798 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:10:12.827710 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:10:12.827710 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:10:12.827710 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:10:12.827710 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:10:12.827710 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 11:10:12.827710 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 11:10:12.827710 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 11:10:12.827710 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 11:10:12.827710 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 11:10:12.827710 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 11:10:12.827710 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 11:10:12.827710 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:10:12.827710 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 628/1263] 10.222.0.44 () {56 vars in 1096 bytes} [Wed Jun 29 11:10:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:10:12.833194 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd056d25-f2b2-49a3-b46f-f3d90b80b63b None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:10:12.851712 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:10:12.929365 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd056d25-f2b2-49a3-b46f-f3d90b80b63b None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:10:12.964315 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f7911d-6625-4f98-9acd-a8745b6f209c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:10:12.964969 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f7911d-6625-4f98-9acd-a8745b6f209c None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:10:12.965248 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f7911d-6625-4f98-9acd-a8745b6f209c None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:10:12.965520 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f7911d-6625-4f98-9acd-a8745b6f209c None admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:10:12.966265 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f7911d-6625-4f98-9acd-a8745b6f209c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:10:12.966568 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f7911d-6625-4f98-9acd-a8745b6f209c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:10:12.968002 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f7911d-6625-4f98-9acd-a8745b6f209c None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:10:12.976695 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 636/1264] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Jun 29 11:10:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:10:12.982402 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b876778-85e6-4b2a-b3ad-8473ea94732d None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:10:13.047496 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:10:13.080073 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b876778-85e6-4b2a-b3ad-8473ea94732d None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:10:13.114004 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e301b7-0582-473d-b317-2675551838b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:10:13.114791 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e301b7-0582-473d-b317-2675551838b4 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:10:13.115070 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e301b7-0582-473d-b317-2675551838b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:10:13.115346 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e301b7-0582-473d-b317-2675551838b4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:10:13.126466 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e301b7-0582-473d-b317-2675551838b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:10:13.127983 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e301b7-0582-473d-b317-2675551838b4 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:10:13.147560 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-48e301b7-0582-473d-b317-2675551838b4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 11:10:13.147560 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:10:13.147560 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:10:13.147560 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:10:13.147560 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:10:13.147560 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:10:13.147560 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:10:13.147560 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:10:13.147560 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:10:13.147560 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:10:13.147560 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:10:13.147560 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:10:13.147560 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 11:10:13.147560 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 11:10:13.147560 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 11:10:13.148771 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 11:10:13.148771 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:10:13.148771 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:10:13.148771 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:10:13.148771 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:10:13.148771 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:10:13.148771 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:10:13.148771 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:10:13.148771 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jun 29 11:10:13.148771 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:10:13.148771 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:10:13.148771 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:10:13.148771 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:10:13.148771 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:10:13.148771 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:10:13.148771 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:10:13.148771 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:10:13.151194 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 11:10:13.151194 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 11:10:13.151194 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 11:10:13.151194 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:10:13.151194 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 629/1265] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jun 29 11:10:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:10:13.154939 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f7911d-6625-4f98-9acd-a8745b6f209c None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:10:13.235469 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:10:13.254986 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f7911d-6625-4f98-9acd-a8745b6f209c None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:10:13.287455 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c41bdd-b674-4f4b-8a4f-20789c1678f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:10:13.288155 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c41bdd-b674-4f4b-8a4f-20789c1678f1 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:10:13.288444 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c41bdd-b674-4f4b-8a4f-20789c1678f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:10:13.288723 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c41bdd-b674-4f4b-8a4f-20789c1678f1 None admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:10:13.289431 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c41bdd-b674-4f4b-8a4f-20789c1678f1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:10:13.289746 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c41bdd-b674-4f4b-8a4f-20789c1678f1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:10:13.291105 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c41bdd-b674-4f4b-8a4f-20789c1678f1 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:10:13.302412 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 637/1266] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Jun 29 11:10:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:10:13.308166 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e301b7-0582-473d-b317-2675551838b4 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:10:13.405279 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e301b7-0582-473d-b317-2675551838b4 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:10:13.424467 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:10:13.439662 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d36261-fdef-4f70-8476-14ff567df78c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:10:13.440510 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d36261-fdef-4f70-8476-14ff567df78c None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:10:13.441285 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d36261-fdef-4f70-8476-14ff567df78c None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:10:13.441566 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d36261-fdef-4f70-8476-14ff567df78c None admin] PATH_INFO: `/v3/users/placement` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:10:13.449539 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d36261-fdef-4f70-8476-14ff567df78c None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:10:13.451907 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d36261-fdef-4f70-8476-14ff567df78c None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:10:13.466434 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-e4d36261-fdef-4f70-8476-14ff567df78c None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 29 11:10:13.466434 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:10:13.466434 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:10:13.466434 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:10:13.466434 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:10:13.466434 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:10:13.466434 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:10:13.466434 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:10:13.466434 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:10:13.466434 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:10:13.466434 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:10:13.466434 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:10:13.466434 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 11:10:13.466434 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 11:10:13.466434 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 11:10:13.466434 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 11:10:13.467600 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:10:13.467600 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:10:13.467600 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 11:10:13.467600 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 11:10:13.467600 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 11:10:13.467600 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 11:10:13.467600 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:10:13.467600 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:10:13.467600 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:10:13.467600 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:10:13.467600 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:10:13.467600 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jun 29 11:10:13.467600 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:10:13.467600 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:10:13.467600 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:10:13.467600 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:10:13.467600 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:10:13.467600 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:10:13.469104 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:10:13.469104 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:10:13.469104 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 11:10:13.469104 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 11:10:13.469104 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 11:10:13.469104 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 11:10:13.469104 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 11:10:13.469104 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 11:10:13.469104 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 29 11:10:13.469104 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:10:13.469104 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 630/1267] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jun 29 11:10:13 2022] GET /identity/v3/users/placement => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:10:13.473476 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c41bdd-b674-4f4b-8a4f-20789c1678f1 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:10:13.576518 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c41bdd-b674-4f4b-8a4f-20789c1678f1 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:10:13.609354 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9874adf9-bf9a-4f38-a3b9-9a760f3a1612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:10:13.610010 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9874adf9-bf9a-4f38-a3b9-9a760f3a1612 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:10:13.610287 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9874adf9-bf9a-4f38-a3b9-9a760f3a1612 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:10:13.610556 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9874adf9-bf9a-4f38-a3b9-9a760f3a1612 None admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:10:13.611376 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9874adf9-bf9a-4f38-a3b9-9a760f3a1612 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:10:13.611682 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9874adf9-bf9a-4f38-a3b9-9a760f3a1612 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:10:13.613747 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9874adf9-bf9a-4f38-a3b9-9a760f3a1612 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:10:13.622018 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:10:13.630514 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 638/1268] 10.222.0.44 () {56 vars in 1143 bytes} [Wed Jun 29 11:10:13 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:10:13.636160 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d36261-fdef-4f70-8476-14ff567df78c None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:10:13.734148 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d36261-fdef-4f70-8476-14ff567df78c None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:10:13.766898 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cc3012-6c4a-4ad6-823a-81b6285ef6d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:10:13.767717 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cc3012-6c4a-4ad6-823a-81b6285ef6d3 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:10:13.768029 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cc3012-6c4a-4ad6-823a-81b6285ef6d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:10:13.768305 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cc3012-6c4a-4ad6-823a-81b6285ef6d3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:10:13.778753 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cc3012-6c4a-4ad6-823a-81b6285ef6d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:10:13.780209 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cc3012-6c4a-4ad6-823a-81b6285ef6d3 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:10:13.796788 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-36cc3012-6c4a-4ad6-823a-81b6285ef6d3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 11:10:13.796788 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:10:13.796788 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:10:13.796788 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:10:13.796788 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:10:13.796788 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:10:13.796788 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:10:13.796788 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:10:13.796788 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:10:13.796788 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:10:13.796788 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:10:13.796788 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:10:13.796788 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 11:10:13.796788 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 11:10:13.796788 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 11:10:13.797881 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 11:10:13.797881 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:10:13.797881 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:10:13.797881 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:10:13.797881 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:10:13.797881 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:10:13.797881 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:10:13.797881 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:10:13.797881 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jun 29 11:10:13.797881 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:10:13.797881 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:10:13.797881 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:10:13.797881 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:10:13.797881 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:10:13.797881 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:10:13.797881 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:10:13.797881 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:10:13.799039 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 11:10:13.799039 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 11:10:13.799039 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 11:10:13.799039 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:10:13.799039 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 631/1269] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jun 29 11:10:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:10:13.804190 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9874adf9-bf9a-4f38-a3b9-9a760f3a1612 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:10:13.901373 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9874adf9-bf9a-4f38-a3b9-9a760f3a1612 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:10:13.935172 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd819eac-d6a9-4a9d-90a2-aefb336459a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:10:13.935886 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd819eac-d6a9-4a9d-90a2-aefb336459a4 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:10:13.936167 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd819eac-d6a9-4a9d-90a2-aefb336459a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:10:13.936438 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd819eac-d6a9-4a9d-90a2-aefb336459a4 None admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:10:13.937144 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd819eac-d6a9-4a9d-90a2-aefb336459a4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:10:13.937446 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd819eac-d6a9-4a9d-90a2-aefb336459a4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:10:13.938794 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd819eac-d6a9-4a9d-90a2-aefb336459a4 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:10:13.950524 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 639/1270] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Jun 29 11:10:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:10:13.956147 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cc3012-6c4a-4ad6-823a-81b6285ef6d3 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:10:13.990235 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:10:14.055713 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cc3012-6c4a-4ad6-823a-81b6285ef6d3 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:10:14.088412 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13944a5-6a86-459f-b2a2-f58b6cb05664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:10:14.090625 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13944a5-6a86-459f-b2a2-f58b6cb05664 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:10:14.090625 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13944a5-6a86-459f-b2a2-f58b6cb05664 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:10:14.090625 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13944a5-6a86-459f-b2a2-f58b6cb05664 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:10:14.096493 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13944a5-6a86-459f-b2a2-f58b6cb05664 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:10:14.098571 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13944a5-6a86-459f-b2a2-f58b6cb05664 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:10:14.112007 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-f13944a5-6a86-459f-b2a2-f58b6cb05664 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 11:10:14.112007 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:10:14.112007 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:10:14.112007 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:10:14.112007 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:10:14.112007 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:10:14.112007 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:10:14.112007 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:10:14.112007 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:10:14.112007 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:10:14.112007 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:10:14.112007 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:10:14.112007 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 11:10:14.112007 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 11:10:14.112007 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 11:10:14.114503 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 11:10:14.114503 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:10:14.114503 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:10:14.114503 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:10:14.114503 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:10:14.114503 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:10:14.114503 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:10:14.114503 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:10:14.114503 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jun 29 11:10:14.114503 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:10:14.114503 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:10:14.114503 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:10:14.114503 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:10:14.114503 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:10:14.114503 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:10:14.114503 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:10:14.114503 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:10:14.115625 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 11:10:14.115625 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 11:10:14.115625 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 11:10:14.115625 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 11:10:14.115625 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 11:10:14.115625 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 11:10:14.115625 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 11:10:14.115625 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:10:14.115625 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 632/1271] 10.222.0.44 () {56 vars in 1111 bytes} [Wed Jun 29 11:10:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:10:14.119644 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd819eac-d6a9-4a9d-90a2-aefb336459a4 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:10:14.179253 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:10:14.220215 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd819eac-d6a9-4a9d-90a2-aefb336459a4 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:10:14.253371 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4293dbdd-7a7e-441b-b611-002a6f845bfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:10:14.254064 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4293dbdd-7a7e-441b-b611-002a6f845bfc None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:10:14.254354 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4293dbdd-7a7e-441b-b611-002a6f845bfc None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:10:14.254636 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4293dbdd-7a7e-441b-b611-002a6f845bfc None admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:10:14.255422 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4293dbdd-7a7e-441b-b611-002a6f845bfc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:10:14.255737 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4293dbdd-7a7e-441b-b611-002a6f845bfc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:10:14.257215 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4293dbdd-7a7e-441b-b611-002a6f845bfc None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:10:14.269032 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 640/1272] 10.222.0.44 () {56 vars in 1148 bytes} [Wed Jun 29 11:10:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:10:14.275114 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13944a5-6a86-459f-b2a2-f58b6cb05664 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:10:14.366943 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:10:14.375677 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13944a5-6a86-459f-b2a2-f58b6cb05664 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:10:14.409528 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3959e25-94e7-49ee-88a3-d681a4c00a20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:10:14.410970 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3959e25-94e7-49ee-88a3-d681a4c00a20 None admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:10:14.411254 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3959e25-94e7-49ee-88a3-d681a4c00a20 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:10:14.411530 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3959e25-94e7-49ee-88a3-d681a4c00a20 None admin] PATH_INFO: `/v3/projects/ebe5dffa4bb8440286687daefb6c98f4/users/1a0505342b704d02977a5125957c2c9e/roles/e95c2ce63e114cc19a9dda6ad21e794f` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:10:14.413508 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3959e25-94e7-49ee-88a3-d681a4c00a20 None admin] RBAC: Authorizing `identity:create_grant(project_id=ebe5dffa4bb8440286687daefb6c98f4, user_id=1a0505342b704d02977a5125957c2c9e, role_id=e95c2ce63e114cc19a9dda6ad21e794f)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:10:14.414946 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3959e25-94e7-49ee-88a3-d681a4c00a20 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:10:14.489333 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 633/1273] 10.222.0.44 () {58 vars in 1439 bytes} [Wed Jun 29 11:10:14 2022] PUT /identity/v3/projects/ebe5dffa4bb8440286687daefb6c98f4/users/1a0505342b704d02977a5125957c2c9e/roles/e95c2ce63e114cc19a9dda6ad21e794f => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:10:16.519981 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf6aa02-8503-4aa5-b9bd-e987803008c5 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:10:16.520535 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf6aa02-8503-4aa5-b9bd-e987803008c5 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:10:16.520535 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf6aa02-8503-4aa5-b9bd-e987803008c5 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:10:16.521943 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 641/1274] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:10:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:10:16.526964 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390fc7fd-021d-4cc1-bca5-72930e8ac2de None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:10:16.527352 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390fc7fd-021d-4cc1-bca5-72930e8ac2de None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:10:16.527522 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390fc7fd-021d-4cc1-bca5-72930e8ac2de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:10:16.543681 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-390fc7fd-021d-4cc1-bca5-72930e8ac2de None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:10:16.544918 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-390fc7fd-021d-4cc1-bca5-72930e8ac2de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:10:16.610826 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 634/1275] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:10:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 11:10:16.635196 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf6aa02-8503-4aa5-b9bd-e987803008c5 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:10:16.736084 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf6aa02-8503-4aa5-b9bd-e987803008c5 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:10:16.753214 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:10:16.769958 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deca44f2-129e-4d8d-8288-49e82cca46be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:10:16.770798 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deca44f2-129e-4d8d-8288-49e82cca46be None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:10:16.771031 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deca44f2-129e-4d8d-8288-49e82cca46be None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:10:16.771423 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deca44f2-129e-4d8d-8288-49e82cca46be None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:10:16.781184 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deca44f2-129e-4d8d-8288-49e82cca46be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 11:10:16.789489 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-deca44f2-129e-4d8d-8288-49e82cca46be None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 11:10:16.789489 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:10:16.789489 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:10:16.789489 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:10:16.789489 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:10:16.789489 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:10:16.789489 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:10:16.789489 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:10:16.789489 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:10:16.789489 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:10:16.789489 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:10:16.789489 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:10:16.789489 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 11:10:16.789489 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 11:10:16.789489 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 11:10:16.789489 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 11:10:16.790603 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 11:10:16.790603 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 11:10:16.790603 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 11:10:16.790603 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 11:10:16.790603 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:10:16.790603 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:10:16.790603 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:10:16.790603 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:10:16.790603 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:10:16.790603 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:10:16.790603 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:10:16.790603 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jun 29 11:10:16.790603 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:10:16.790603 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:10:16.790603 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:10:16.790603 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:10:16.791642 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:10:16.791642 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:10:16.791642 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:10:16.791642 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:10:16.791642 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 11:10:16.791642 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 11:10:16.791642 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 11:10:16.791642 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 11:10:16.791642 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 11:10:16.791642 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 11:10:16.791642 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 11:10:16.791642 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jun 29 11:10:16.791642 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 642/1276] 10.222.0.44 () {56 vars in 1096 bytes} [Wed Jun 29 11:10:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:10:16.796717 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390fc7fd-021d-4cc1-bca5-72930e8ac2de None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:10:16.880227 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:10:16.902583 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390fc7fd-021d-4cc1-bca5-72930e8ac2de None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:10:16.936761 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da660828-0b19-498d-a9c3-4fa3758b7ce6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:10:16.937761 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da660828-0b19-498d-a9c3-4fa3758b7ce6 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:10:16.938397 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da660828-0b19-498d-a9c3-4fa3758b7ce6 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:10:16.938758 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da660828-0b19-498d-a9c3-4fa3758b7ce6 None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:10:16.939508 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da660828-0b19-498d-a9c3-4fa3758b7ce6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:10:16.939859 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da660828-0b19-498d-a9c3-4fa3758b7ce6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:10:16.941266 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da660828-0b19-498d-a9c3-4fa3758b7ce6 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:10:16.949478 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 635/1277] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Jun 29 11:10:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:10:16.955274 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deca44f2-129e-4d8d-8288-49e82cca46be None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:10:17.053912 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deca44f2-129e-4d8d-8288-49e82cca46be None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:10:17.087477 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5043bc21-4c6e-4622-92df-db0ff6ed9cfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:10:17.088282 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5043bc21-4c6e-4622-92df-db0ff6ed9cfb None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:10:17.088563 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5043bc21-4c6e-4622-92df-db0ff6ed9cfb None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:10:17.088834 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5043bc21-4c6e-4622-92df-db0ff6ed9cfb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:10:17.098919 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5043bc21-4c6e-4622-92df-db0ff6ed9cfb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:10:17.100375 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5043bc21-4c6e-4622-92df-db0ff6ed9cfb None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:10:17.116885 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-5043bc21-4c6e-4622-92df-db0ff6ed9cfb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 11:10:17.116885 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:10:17.116885 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:10:17.116885 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:10:17.116885 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:10:17.116885 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:10:17.116885 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:10:17.116885 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:10:17.116885 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:10:17.116885 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:10:17.116885 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:10:17.116885 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:10:17.116885 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 11:10:17.116885 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 11:10:17.116885 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 11:10:17.117965 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 11:10:17.117965 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:10:17.117965 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:10:17.117965 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:10:17.117965 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:10:17.117965 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:10:17.117965 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:10:17.117965 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:10:17.117965 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jun 29 11:10:17.117965 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:10:17.117965 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:10:17.117965 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:10:17.117965 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:10:17.117965 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:10:17.117965 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:10:17.117965 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:10:17.117965 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:10:17.119066 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 11:10:17.119066 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 11:10:17.119066 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 11:10:17.119066 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jun 29 11:10:17.119066 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 643/1278] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jun 29 11:10:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:10:17.123922 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da660828-0b19-498d-a9c3-4fa3758b7ce6 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:10:17.220239 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da660828-0b19-498d-a9c3-4fa3758b7ce6 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:10:17.251616 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149a671a-8de9-4827-bb05-933af4b36b04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:10:17.252282 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149a671a-8de9-4827-bb05-933af4b36b04 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:10:17.252564 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149a671a-8de9-4827-bb05-933af4b36b04 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:10:17.252842 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149a671a-8de9-4827-bb05-933af4b36b04 None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:10:17.254064 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149a671a-8de9-4827-bb05-933af4b36b04 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:10:17.254365 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149a671a-8de9-4827-bb05-933af4b36b04 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:10:17.255694 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149a671a-8de9-4827-bb05-933af4b36b04 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:10:17.266990 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 636/1279] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Jun 29 11:10:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:10:17.273255 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5043bc21-4c6e-4622-92df-db0ff6ed9cfb None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:10:17.303322 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:10:17.370219 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5043bc21-4c6e-4622-92df-db0ff6ed9cfb None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:10:17.402862 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759ff727-823d-47cb-b928-fa052bba428b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:10:17.403667 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759ff727-823d-47cb-b928-fa052bba428b None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:10:17.403980 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759ff727-823d-47cb-b928-fa052bba428b None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:10:17.404255 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759ff727-823d-47cb-b928-fa052bba428b None admin] PATH_INFO: `/v3/users/placement` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:10:17.411467 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759ff727-823d-47cb-b928-fa052bba428b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:10:17.419333 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759ff727-823d-47cb-b928-fa052bba428b None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:10:17.427380 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-759ff727-823d-47cb-b928-fa052bba428b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 29 11:10:17.427380 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:10:17.427380 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:10:17.427380 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:10:17.427380 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:10:17.427380 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:10:17.427380 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:10:17.427380 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:10:17.427380 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:10:17.427380 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:10:17.427380 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:10:17.427380 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:10:17.427380 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 11:10:17.427380 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 11:10:17.427380 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 11:10:17.427380 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 11:10:17.428554 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:10:17.428554 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:10:17.428554 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 11:10:17.428554 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 11:10:17.428554 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 11:10:17.428554 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 11:10:17.428554 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:10:17.428554 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:10:17.428554 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:10:17.428554 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:10:17.428554 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:10:17.428554 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jun 29 11:10:17.428554 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:10:17.428554 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:10:17.428554 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:10:17.428554 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:10:17.428554 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:10:17.428554 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:10:17.429786 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:10:17.429786 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:10:17.429786 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 11:10:17.429786 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 11:10:17.429786 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 11:10:17.429786 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 11:10:17.429786 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 11:10:17.429786 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 11:10:17.429786 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 29 11:10:17.429786 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jun 29 11:10:17.429786 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 644/1280] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jun 29 11:10:17 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:10:17.434384 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149a671a-8de9-4827-bb05-933af4b36b04 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:10:17.438189 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:10:17.544553 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149a671a-8de9-4827-bb05-933af4b36b04 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:10:17.579658 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c740acad-7d53-411c-86fd-2e3245d172c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:10:17.580357 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c740acad-7d53-411c-86fd-2e3245d172c8 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:10:17.580642 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c740acad-7d53-411c-86fd-2e3245d172c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:10:17.580917 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c740acad-7d53-411c-86fd-2e3245d172c8 None admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:10:17.581747 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c740acad-7d53-411c-86fd-2e3245d172c8 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:10:17.582053 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c740acad-7d53-411c-86fd-2e3245d172c8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:10:17.583462 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c740acad-7d53-411c-86fd-2e3245d172c8 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:10:17.600123 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 637/1281] 10.222.0.44 () {56 vars in 1143 bytes} [Wed Jun 29 11:10:17 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:10:17.605874 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759ff727-823d-47cb-b928-fa052bba428b None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:10:17.694661 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:10:17.704158 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759ff727-823d-47cb-b928-fa052bba428b None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:10:17.736490 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4a76c6-f32f-4a00-be18-d2e187ee54ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:10:17.737147 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4a76c6-f32f-4a00-be18-d2e187ee54ec None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:10:17.737428 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4a76c6-f32f-4a00-be18-d2e187ee54ec None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:10:17.738267 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4a76c6-f32f-4a00-be18-d2e187ee54ec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:10:17.803415 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4a76c6-f32f-4a00-be18-d2e187ee54ec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:10:17.804871 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4a76c6-f32f-4a00-be18-d2e187ee54ec None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:10:17.888843 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:10:17.898443 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 645/1282] 10.222.0.44 () {58 vars in 1282 bytes} [Wed Jun 29 11:10:17 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 29 11:10:17.904307 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c740acad-7d53-411c-86fd-2e3245d172c8 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:10:17.965959 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:10:18.005079 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c740acad-7d53-411c-86fd-2e3245d172c8 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:10:18.037315 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80280c3-e2ea-43bc-ab32-76db03bcfabd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:10:18.038092 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80280c3-e2ea-43bc-ab32-76db03bcfabd None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:10:18.038373 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80280c3-e2ea-43bc-ab32-76db03bcfabd None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:10:18.038648 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80280c3-e2ea-43bc-ab32-76db03bcfabd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:10:18.048767 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80280c3-e2ea-43bc-ab32-76db03bcfabd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:10:18.050205 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80280c3-e2ea-43bc-ab32-76db03bcfabd None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:10:18.067462 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-f80280c3-e2ea-43bc-ab32-76db03bcfabd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 11:10:18.067462 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:10:18.067462 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:10:18.067462 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:10:18.067462 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:10:18.067462 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:10:18.067462 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:10:18.067462 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:10:18.067462 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:10:18.067462 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:10:18.067462 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:10:18.067462 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:10:18.067462 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 11:10:18.067462 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 11:10:18.067462 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 11:10:18.071515 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 11:10:18.071515 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:10:18.071515 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:10:18.071515 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:10:18.071515 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:10:18.071515 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:10:18.071515 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:10:18.071515 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:10:18.071515 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jun 29 11:10:18.071515 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:10:18.071515 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:10:18.071515 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:10:18.071515 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:10:18.071515 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:10:18.071515 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:10:18.071515 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:10:18.071515 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:10:18.072512 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 11:10:18.072512 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 11:10:18.072512 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 11:10:18.072512 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:10:18.072512 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 638/1283] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jun 29 11:10:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:10:18.074626 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4a76c6-f32f-4a00-be18-d2e187ee54ec None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:10:18.170732 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4a76c6-f32f-4a00-be18-d2e187ee54ec None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:10:18.204714 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb2afc6-dd92-4534-897c-89f278ab5e17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:10:18.205366 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb2afc6-dd92-4534-897c-89f278ab5e17 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:10:18.205649 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb2afc6-dd92-4534-897c-89f278ab5e17 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:10:18.205926 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb2afc6-dd92-4534-897c-89f278ab5e17 None admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:10:18.206658 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb2afc6-dd92-4534-897c-89f278ab5e17 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:10:18.206962 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb2afc6-dd92-4534-897c-89f278ab5e17 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:10:18.208355 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb2afc6-dd92-4534-897c-89f278ab5e17 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:10:18.219322 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 646/1284] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Jun 29 11:10:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:10:18.225011 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80280c3-e2ea-43bc-ab32-76db03bcfabd None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:10:18.324444 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80280c3-e2ea-43bc-ab32-76db03bcfabd None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:10:18.338283 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:10:18.358184 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d24cc97-0e2f-4fb6-bd3a-69ce5a851e53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:10:18.358970 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d24cc97-0e2f-4fb6-bd3a-69ce5a851e53 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:10:18.359257 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d24cc97-0e2f-4fb6-bd3a-69ce5a851e53 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:10:18.359534 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d24cc97-0e2f-4fb6-bd3a-69ce5a851e53 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:10:18.366022 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d24cc97-0e2f-4fb6-bd3a-69ce5a851e53 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:10:18.367442 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d24cc97-0e2f-4fb6-bd3a-69ce5a851e53 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:10:18.381529 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-3d24cc97-0e2f-4fb6-bd3a-69ce5a851e53 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 11:10:18.381529 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:10:18.381529 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:10:18.381529 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:10:18.381529 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:10:18.381529 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:10:18.381529 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:10:18.381529 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:10:18.381529 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:10:18.381529 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:10:18.381529 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:10:18.381529 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:10:18.381529 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 11:10:18.381529 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 11:10:18.381529 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 11:10:18.382553 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 11:10:18.382553 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:10:18.382553 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:10:18.382553 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:10:18.382553 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:10:18.382553 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:10:18.382553 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:10:18.382553 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:10:18.382553 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( Jun 29 11:10:18.382553 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:10:18.382553 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:10:18.382553 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:10:18.382553 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:10:18.382553 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:10:18.382553 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:10:18.382553 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:10:18.382553 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:10:18.383456 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 11:10:18.383456 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 11:10:18.383456 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 11:10:18.383456 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 11:10:18.383456 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 11:10:18.383456 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 11:10:18.383456 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 11:10:18.383456 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Jun 29 11:10:18.383456 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 639/1285] 10.222.0.44 () {56 vars in 1111 bytes} [Wed Jun 29 11:10:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:10:18.388614 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb2afc6-dd92-4534-897c-89f278ab5e17 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:10:18.438125 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:10:18.489100 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb2afc6-dd92-4534-897c-89f278ab5e17 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:10:18.521442 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89e4c53-d8dc-49c4-8758-9d6a77d70628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:10:18.522087 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89e4c53-d8dc-49c4-8758-9d6a77d70628 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:10:18.522368 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89e4c53-d8dc-49c4-8758-9d6a77d70628 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:10:18.522643 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89e4c53-d8dc-49c4-8758-9d6a77d70628 None admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:10:18.525208 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89e4c53-d8dc-49c4-8758-9d6a77d70628 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:10:18.525208 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89e4c53-d8dc-49c4-8758-9d6a77d70628 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:10:18.525208 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89e4c53-d8dc-49c4-8758-9d6a77d70628 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:10:18.536549 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 647/1286] 10.222.0.44 () {56 vars in 1148 bytes} [Wed Jun 29 11:10:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:10:18.542575 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d24cc97-0e2f-4fb6-bd3a-69ce5a851e53 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:10:18.640041 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d24cc97-0e2f-4fb6-bd3a-69ce5a851e53 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:10:18.673221 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b88a41-6982-4e0d-a0a0-759d177c9e15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:10:18.673866 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b88a41-6982-4e0d-a0a0-759d177c9e15 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:10:18.674154 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b88a41-6982-4e0d-a0a0-759d177c9e15 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:10:18.674432 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b88a41-6982-4e0d-a0a0-759d177c9e15 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:10:18.675801 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b88a41-6982-4e0d-a0a0-759d177c9e15 None admin] RBAC: Adding query filter params (role.id=e95c2ce63e114cc19a9dda6ad21e794f, scope.project.id=ebe5dffa4bb8440286687daefb6c98f4, user.id=1a0505342b704d02977a5125957c2c9e) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:10:18.676144 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b88a41-6982-4e0d-a0a0-759d177c9e15 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:10:18.677557 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b88a41-6982-4e0d-a0a0-759d177c9e15 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:10:18.683220 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 640/1287] 10.222.0.44 () {56 vars in 1374 bytes} [Wed Jun 29 11:10:18 2022] GET /identity/v3/role_assignments?user.id=1a0505342b704d02977a5125957c2c9e&scope.project.id=ebe5dffa4bb8440286687daefb6c98f4&role.id=e95c2ce63e114cc19a9dda6ad21e794f => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:10:20.706817 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1651809b-6067-483c-8392-57458f68bf7e None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:10:20.707411 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1651809b-6067-483c-8392-57458f68bf7e None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:10:20.707411 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1651809b-6067-483c-8392-57458f68bf7e None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:10:20.708750 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 648/1288] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:10:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:10:20.713962 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a982100b-813d-4326-957a-0bc1db1ad822 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:10:20.714423 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a982100b-813d-4326-957a-0bc1db1ad822 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:10:20.714515 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a982100b-813d-4326-957a-0bc1db1ad822 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:10:20.731593 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-a982100b-813d-4326-957a-0bc1db1ad822 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:10:20.732845 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-a982100b-813d-4326-957a-0bc1db1ad822 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:10:20.765259 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:10:20.802385 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 641/1289] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:10:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 11:10:20.824800 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1651809b-6067-483c-8392-57458f68bf7e None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:10:20.929268 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1651809b-6067-483c-8392-57458f68bf7e None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:10:20.940313 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:10:20.962763 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d11ff0d-7bff-4be5-a75b-43bcddd68e82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:10:20.963492 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d11ff0d-7bff-4be5-a75b-43bcddd68e82 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:10:20.963775 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d11ff0d-7bff-4be5-a75b-43bcddd68e82 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:10:20.964077 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d11ff0d-7bff-4be5-a75b-43bcddd68e82 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:10:20.964725 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d11ff0d-7bff-4be5-a75b-43bcddd68e82 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:10:20.966084 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d11ff0d-7bff-4be5-a75b-43bcddd68e82 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:10:20.978110 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-9d11ff0d-7bff-4be5-a75b-43bcddd68e82 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 29 11:10:20.978110 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:10:20.978110 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 29 11:10:20.978110 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 29 11:10:20.978110 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 29 11:10:20.978110 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 29 11:10:20.978110 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 29 11:10:20.978110 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 11:10:20.978110 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 29 11:10:20.978110 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jun 29 11:10:20.978110 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 29 11:10:20.978110 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jun 29 11:10:20.978110 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:10:20.978110 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:10:20.978110 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:10:20.978110 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:10:20.978110 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:10:20.979325 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:10:20.979325 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:10:20.979325 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:10:20.979325 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:10:20.979325 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:10:20.979325 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:10:20.979325 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 29 11:10:20.979325 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 29 11:10:20.979325 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 29 11:10:20.979325 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 29 11:10:20.979325 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:10:20.979325 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:10:20.979325 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:10:20.979325 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:10:20.979325 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:10:20.979325 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:10:20.980606 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:10:20.980606 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jun 29 11:10:20.980606 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:10:20.980606 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:10:20.980606 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:10:20.980606 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:10:20.980606 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:10:20.980606 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:10:20.980606 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:10:20.980606 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:10:20.980606 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 29 11:10:20.980606 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 11:10:20.980606 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 29 11:10:20.980606 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jun 29 11:10:20.980606 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 649/1290] 10.222.0.44 () {56 vars in 1117 bytes} [Wed Jun 29 11:10:20 2022] GET /identity/v3/services/placement => generated 90 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:10:20.985191 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a982100b-813d-4326-957a-0bc1db1ad822 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:10:21.080617 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a982100b-813d-4326-957a-0bc1db1ad822 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:10:21.113962 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f963bc8-70bb-4124-af47-a619891dddc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:10:21.114625 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f963bc8-70bb-4124-af47-a619891dddc0 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:10:21.114907 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f963bc8-70bb-4124-af47-a619891dddc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:10:21.115184 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f963bc8-70bb-4124-af47-a619891dddc0 None admin] PATH_INFO: `/v3/services` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:10:21.115961 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f963bc8-70bb-4124-af47-a619891dddc0 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:10:21.116288 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f963bc8-70bb-4124-af47-a619891dddc0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:10:21.117642 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f963bc8-70bb-4124-af47-a619891dddc0 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:10:21.122422 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 642/1291] 10.222.0.44 () {56 vars in 1116 bytes} [Wed Jun 29 11:10:20 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:10:21.128250 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d11ff0d-7bff-4be5-a75b-43bcddd68e82 None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:10:21.229902 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d11ff0d-7bff-4be5-a75b-43bcddd68e82 None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:10:21.262322 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71088bed-b888-4c18-8a29-4c4c9a580bed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:10:21.262975 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71088bed-b888-4c18-8a29-4c4c9a580bed None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:10:21.263258 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71088bed-b888-4c18-8a29-4c4c9a580bed None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:10:21.263532 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71088bed-b888-4c18-8a29-4c4c9a580bed None admin] PATH_INFO: `/v3/services` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:10:21.264260 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71088bed-b888-4c18-8a29-4c4c9a580bed None admin] RBAC: Adding query filter params (type=placement) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:10:21.264656 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71088bed-b888-4c18-8a29-4c4c9a580bed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:10:21.266010 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71088bed-b888-4c18-8a29-4c4c9a580bed None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:10:21.270900 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 650/1292] 10.222.0.44 () {56 vars in 1116 bytes} [Wed Jun 29 11:10:21 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:10:23.256610 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2c9165-0c7b-4532-9c72-e6137b035d52 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:10:23.257167 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2c9165-0c7b-4532-9c72-e6137b035d52 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:10:23.257167 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2c9165-0c7b-4532-9c72-e6137b035d52 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:10:23.258569 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 643/1293] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:10:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:10:23.263689 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8084f5f2-61ba-4e8f-ba4d-24b8dd6b6388 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:10:23.264129 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8084f5f2-61ba-4e8f-ba4d-24b8dd6b6388 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:10:23.264344 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8084f5f2-61ba-4e8f-ba4d-24b8dd6b6388 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:10:23.286776 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-8084f5f2-61ba-4e8f-ba4d-24b8dd6b6388 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:10:23.287975 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-8084f5f2-61ba-4e8f-ba4d-24b8dd6b6388 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:10:23.317420 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:10:23.353463 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 651/1294] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:10:23 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 11:10:23.376808 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2c9165-0c7b-4532-9c72-e6137b035d52 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:10:23.407469 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:10:23.479221 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2c9165-0c7b-4532-9c72-e6137b035d52 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:10:23.516586 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9649f58c-583c-4626-a3cd-95a2f704ee87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:10:23.517180 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9649f58c-583c-4626-a3cd-95a2f704ee87 None admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:10:23.517466 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9649f58c-583c-4626-a3cd-95a2f704ee87 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:10:23.517745 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9649f58c-583c-4626-a3cd-95a2f704ee87 None admin] PATH_INFO: `/v3/services` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:10:23.518459 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9649f58c-583c-4626-a3cd-95a2f704ee87 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:10:23.520406 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9649f58c-583c-4626-a3cd-95a2f704ee87 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:10:23.592796 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 644/1295] 10.222.0.44 () {60 vars in 1141 bytes} [Wed Jun 29 11:10:23 2022] POST /identity/v3/services => generated 246 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:10:25.660576 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38781eed-4fee-4ecc-9a1e-bf22c009ae70 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:10:25.661137 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38781eed-4fee-4ecc-9a1e-bf22c009ae70 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:10:25.661254 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38781eed-4fee-4ecc-9a1e-bf22c009ae70 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:10:25.662548 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 652/1296] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:10:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:10:25.667602 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535b84d9-6d9b-4acb-a468-4a7c4093ad5e None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:10:25.668004 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535b84d9-6d9b-4acb-a468-4a7c4093ad5e None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:10:25.668191 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535b84d9-6d9b-4acb-a468-4a7c4093ad5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:10:25.685011 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-535b84d9-6d9b-4acb-a468-4a7c4093ad5e None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:10:25.686190 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-535b84d9-6d9b-4acb-a468-4a7c4093ad5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:10:25.758502 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 645/1297] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:10:25 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 11:10:25.781089 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38781eed-4fee-4ecc-9a1e-bf22c009ae70 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:10:25.880363 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38781eed-4fee-4ecc-9a1e-bf22c009ae70 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:10:25.912696 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b980d23e-3021-4f20-a99a-03cda95cbc4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:10:25.913432 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b980d23e-3021-4f20-a99a-03cda95cbc4e None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:10:25.913718 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b980d23e-3021-4f20-a99a-03cda95cbc4e None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:10:25.913996 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b980d23e-3021-4f20-a99a-03cda95cbc4e None admin] PATH_INFO: `/v3/services/placement` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:10:25.914652 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b980d23e-3021-4f20-a99a-03cda95cbc4e None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:10:25.916111 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b980d23e-3021-4f20-a99a-03cda95cbc4e None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:10:25.929263 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-b980d23e-3021-4f20-a99a-03cda95cbc4e None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 29 11:10:25.929263 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:10:25.929263 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 29 11:10:25.929263 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 29 11:10:25.929263 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 29 11:10:25.929263 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 29 11:10:25.929263 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 29 11:10:25.929263 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 11:10:25.929263 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 29 11:10:25.929263 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jun 29 11:10:25.929263 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 29 11:10:25.929263 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jun 29 11:10:25.929263 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:10:25.929263 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:10:25.929263 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:10:25.929263 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:10:25.929263 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:10:25.930564 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:10:25.930564 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:10:25.930564 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:10:25.930564 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:10:25.930564 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:10:25.930564 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:10:25.930564 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 29 11:10:25.930564 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 29 11:10:25.930564 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 29 11:10:25.930564 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 29 11:10:25.930564 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:10:25.930564 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:10:25.930564 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:10:25.930564 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:10:25.930564 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:10:25.930564 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:10:25.931611 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:10:25.931611 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jun 29 11:10:25.931611 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:10:25.931611 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:10:25.931611 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:10:25.931611 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:10:25.931611 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:10:25.931611 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:10:25.931611 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:10:25.931611 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:10:25.931611 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 29 11:10:25.931611 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 11:10:25.931611 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 29 11:10:25.931611 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jun 29 11:10:25.931611 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 653/1298] 10.222.0.44 () {56 vars in 1117 bytes} [Wed Jun 29 11:10:25 2022] GET /identity/v3/services/placement => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:10:25.938087 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535b84d9-6d9b-4acb-a468-4a7c4093ad5e None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:10:25.955669 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:10:26.032261 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535b84d9-6d9b-4acb-a468-4a7c4093ad5e None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:10:26.066715 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4da543-624c-405f-8a4b-a5f5aa76f1d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:10:26.067379 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4da543-624c-405f-8a4b-a5f5aa76f1d8 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:10:26.067666 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4da543-624c-405f-8a4b-a5f5aa76f1d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:10:26.067970 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4da543-624c-405f-8a4b-a5f5aa76f1d8 None admin] PATH_INFO: `/v3/services` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:10:26.068669 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4da543-624c-405f-8a4b-a5f5aa76f1d8 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:10:26.068976 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4da543-624c-405f-8a4b-a5f5aa76f1d8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:10:26.070360 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4da543-624c-405f-8a4b-a5f5aa76f1d8 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:10:26.075696 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 646/1299] 10.222.0.44 () {56 vars in 1116 bytes} [Wed Jun 29 11:10:25 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:10:26.082461 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b980d23e-3021-4f20-a99a-03cda95cbc4e None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:10:26.113660 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:10:26.179437 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b980d23e-3021-4f20-a99a-03cda95cbc4e None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:10:26.213477 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7172b48-d1ce-44d4-a2fb-be471bf42ce2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:10:26.214124 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7172b48-d1ce-44d4-a2fb-be471bf42ce2 None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:10:26.214407 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7172b48-d1ce-44d4-a2fb-be471bf42ce2 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:10:26.214683 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7172b48-d1ce-44d4-a2fb-be471bf42ce2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:10:26.215461 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7172b48-d1ce-44d4-a2fb-be471bf42ce2 None admin] RBAC: Adding query filter params (interface=public, service_id=761d991273e14b459b2d9ac100de774b, region_id=RegionOne) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:10:26.215771 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7172b48-d1ce-44d4-a2fb-be471bf42ce2 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:10:26.217161 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7172b48-d1ce-44d4-a2fb-be471bf42ce2 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:10:26.222094 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 654/1300] 10.222.0.44 () {56 vars in 1251 bytes} [Wed Jun 29 11:10:26 2022] GET /identity/v3/endpoints?service_id=761d991273e14b459b2d9ac100de774b&interface=public®ion_id=RegionOne => generated 194 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:10:26.227750 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4da543-624c-405f-8a4b-a5f5aa76f1d8 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:10:26.306326 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:10:26.324167 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4da543-624c-405f-8a4b-a5f5aa76f1d8 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:10:26.356836 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f9495d-0dca-455e-8f2b-89f52b64344b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:10:26.357700 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f9495d-0dca-455e-8f2b-89f52b64344b None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:10:26.358384 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f9495d-0dca-455e-8f2b-89f52b64344b None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:10:26.358668 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f9495d-0dca-455e-8f2b-89f52b64344b None admin] PATH_INFO: `/v3/services` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:10:26.359344 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f9495d-0dca-455e-8f2b-89f52b64344b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:10:26.360791 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f9495d-0dca-455e-8f2b-89f52b64344b None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:10:26.368183 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 647/1301] 10.222.0.44 () {56 vars in 1087 bytes} [Wed Jun 29 11:10:26 2022] GET /identity/v3/services => generated 2191 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 11:10:28.400856 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2923c20-3f24-4add-9510-be14a81bb9b3 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:10:28.401380 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2923c20-3f24-4add-9510-be14a81bb9b3 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:10:28.401496 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2923c20-3f24-4add-9510-be14a81bb9b3 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:10:28.402766 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 655/1302] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:10:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:10:28.407964 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a24d2b0-2e3a-408b-8faa-d31c484f8604 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:10:28.408369 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a24d2b0-2e3a-408b-8faa-d31c484f8604 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:10:28.408516 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a24d2b0-2e3a-408b-8faa-d31c484f8604 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:10:28.425937 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-5a24d2b0-2e3a-408b-8faa-d31c484f8604 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:10:28.427148 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-5a24d2b0-2e3a-408b-8faa-d31c484f8604 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:10:28.491964 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 648/1303] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:10:28 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 11:10:28.514570 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2923c20-3f24-4add-9510-be14a81bb9b3 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:10:28.614745 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:10:28.621566 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2923c20-3f24-4add-9510-be14a81bb9b3 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:10:28.655422 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787d0163-03ad-4444-b2be-0cc8af1fd27b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:10:28.656169 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787d0163-03ad-4444-b2be-0cc8af1fd27b None admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:10:28.656450 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787d0163-03ad-4444-b2be-0cc8af1fd27b None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:10:28.656722 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787d0163-03ad-4444-b2be-0cc8af1fd27b None admin] PATH_INFO: `/v3/services/placement` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:10:28.657363 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787d0163-03ad-4444-b2be-0cc8af1fd27b None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:10:28.658768 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787d0163-03ad-4444-b2be-0cc8af1fd27b None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:10:28.670592 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application [None req-787d0163-03ad-4444-b2be-0cc8af1fd27b None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 29 11:10:28.670592 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:10:28.670592 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 29 11:10:28.670592 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 29 11:10:28.670592 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 29 11:10:28.670592 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 29 11:10:28.670592 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 29 11:10:28.670592 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 11:10:28.670592 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 29 11:10:28.670592 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jun 29 11:10:28.670592 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 29 11:10:28.670592 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jun 29 11:10:28.670592 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 11:10:28.670592 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 11:10:28.670592 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 11:10:28.670592 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 11:10:28.670592 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 11:10:28.671733 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 11:10:28.671733 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 11:10:28.671733 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 11:10:28.671733 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 11:10:28.671733 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 11:10:28.671733 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 11:10:28.671733 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 29 11:10:28.671733 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 29 11:10:28.671733 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 29 11:10:28.671733 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 29 11:10:28.671733 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 11:10:28.671733 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 11:10:28.671733 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 11:10:28.671733 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 11:10:28.671733 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 11:10:28.671733 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 11:10:28.672811 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 11:10:28.672811 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application with Lock( Jun 29 11:10:28.672811 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 11:10:28.672811 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self._enter() Jun 29 11:10:28.672811 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 11:10:28.672811 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 11:10:28.672811 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 11:10:28.672811 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application return self.creator() Jun 29 11:10:28.672811 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 11:10:28.672811 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application created_value = creator( Jun 29 11:10:28.672811 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 29 11:10:28.672811 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 11:10:28.672811 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 29 11:10:28.672811 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: ERROR keystone.server.flask.application Jun 29 11:10:28.672811 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 656/1304] 10.222.0.44 () {56 vars in 1117 bytes} [Wed Jun 29 11:10:28 2022] GET /identity/v3/services/placement => generated 90 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 11:10:28.678462 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a24d2b0-2e3a-408b-8faa-d31c484f8604 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:10:28.753401 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:10:28.774999 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a24d2b0-2e3a-408b-8faa-d31c484f8604 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:10:28.807632 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6ed954-f648-4421-aa2d-38563a8c76fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:10:28.808324 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6ed954-f648-4421-aa2d-38563a8c76fa None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:10:28.808608 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6ed954-f648-4421-aa2d-38563a8c76fa None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:10:28.808886 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6ed954-f648-4421-aa2d-38563a8c76fa None admin] PATH_INFO: `/v3/services` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:10:28.809582 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6ed954-f648-4421-aa2d-38563a8c76fa None admin] RBAC: Adding query filter params (name=placement) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:10:28.809889 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6ed954-f648-4421-aa2d-38563a8c76fa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:10:28.811243 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6ed954-f648-4421-aa2d-38563a8c76fa None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:10:28.816920 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 649/1305] 10.222.0.44 () {56 vars in 1116 bytes} [Wed Jun 29 11:10:28 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:10:28.822985 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787d0163-03ad-4444-b2be-0cc8af1fd27b None admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:10:28.921381 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787d0163-03ad-4444-b2be-0cc8af1fd27b None admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:10:28.954640 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad779aa3-8a04-40ed-8f27-0fd597ea1337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:10:28.955296 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad779aa3-8a04-40ed-8f27-0fd597ea1337 None admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:10:28.955583 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad779aa3-8a04-40ed-8f27-0fd597ea1337 None admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:10:28.955911 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad779aa3-8a04-40ed-8f27-0fd597ea1337 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:10:28.956638 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad779aa3-8a04-40ed-8f27-0fd597ea1337 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:10:28.958002 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad779aa3-8a04-40ed-8f27-0fd597ea1337 None admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:10:29.059959 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 657/1306] 10.222.0.44 () {60 vars in 1144 bytes} [Wed Jun 29 11:10:28 2022] POST /identity/v3/endpoints => generated 331 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:10:38.837497 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=30 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:10:39.055494 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:16:24.358420 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4ef53c-8228-46a1-9932-ec0116503e63 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:16:24.358897 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4ef53c-8228-46a1-9932-ec0116503e63 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:16:24.358976 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4ef53c-8228-46a1-9932-ec0116503e63 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:16:24.366353 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff61761-74fe-4425-9f98-b47d00b11f62 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:16:24.366730 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff61761-74fe-4425-9f98-b47d00b11f62 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:16:24.366901 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff61761-74fe-4425-9f98-b47d00b11f62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:16:24.370420 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 650/1307] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:16:24 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:16:24.501995 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-0ff61761-74fe-4425-9f98-b47d00b11f62 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:16:24.504085 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-0ff61761-74fe-4425-9f98-b47d00b11f62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:16:24.649197 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 658/1308] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:16:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:16:24.702569 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c490b3-ae47-4054-92a6-3213ffe202db None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:16:24.702755 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c490b3-ae47-4054-92a6-3213ffe202db None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:16:24.703073 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c490b3-ae47-4054-92a6-3213ffe202db None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:16:24.705293 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 651/1309] 10.222.0.44 () {54 vars in 964 bytes} [Wed Jun 29 11:16:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:16:24.713102 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd37496-f98c-4003-a8a2-c2440a08207a None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:16:24.713660 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd37496-f98c-4003-a8a2-c2440a08207a None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:16:24.714090 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd37496-f98c-4003-a8a2-c2440a08207a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:16:24.802748 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-1fd37496-f98c-4003-a8a2-c2440a08207a None None] MFA Rules not processed for user `b5f4bb9f989e4024a5da28dced646b92`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:16:24.804002 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-1fd37496-f98c-4003-a8a2-c2440a08207a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:16:24.831317 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:16:24.888401 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 659/1310] 10.222.0.44 () {58 vars in 1061 bytes} [Wed Jun 29 11:16:24 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:16:24.897923 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c490b3-ae47-4054-92a6-3213ffe202db None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:16:24.899883 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-05c490b3-ae47-4054-92a6-3213ffe202db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:16:25.023377 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c490b3-ae47-4054-92a6-3213ffe202db None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:16:25.100969 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df59d9a7-aff0-45b3-9e18-56b539b0510a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f4bb9f989e4024a5da28dced646b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:16:25.101714 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df59d9a7-aff0-45b3-9e18-56b539b0510a service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:16:25.102067 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df59d9a7-aff0-45b3-9e18-56b539b0510a service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:16:25.102407 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df59d9a7-aff0-45b3-9e18-56b539b0510a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:16:25.104154 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-df59d9a7-aff0-45b3-9e18-56b539b0510a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:16:25.166729 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:16:25.226338 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df59d9a7-aff0-45b3-9e18-56b539b0510a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:16:25.227780 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df59d9a7-aff0-45b3-9e18-56b539b0510a service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:16:25.247542 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 652/1311] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:16:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:16:27.581415 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7abe57-2e86-4fbd-a3c8-e10236d83e4a None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:16:27.581960 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7abe57-2e86-4fbd-a3c8-e10236d83e4a None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:16:27.581960 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7abe57-2e86-4fbd-a3c8-e10236d83e4a None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:16:27.583488 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 660/1312] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:16:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:16:27.588900 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140396d3-e6e9-4db5-b9c9-f884a8c25968 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:16:27.590587 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140396d3-e6e9-4db5-b9c9-f884a8c25968 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:16:27.590980 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140396d3-e6e9-4db5-b9c9-f884a8c25968 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:16:27.673652 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-140396d3-e6e9-4db5-b9c9-f884a8c25968 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:16:27.674877 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-140396d3-e6e9-4db5-b9c9-f884a8c25968 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:16:27.798794 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 653/1313] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:16:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:16:27.839962 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7abe57-2e86-4fbd-a3c8-e10236d83e4a None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:16:27.841835 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-2e7abe57-2e86-4fbd-a3c8-e10236d83e4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:16:27.960031 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7abe57-2e86-4fbd-a3c8-e10236d83e4a None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:16:28.037577 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:16:28.040544 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ced6542-e29a-41fc-a2dd-8fe4ee875edf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f4bb9f989e4024a5da28dced646b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:16:28.041203 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ced6542-e29a-41fc-a2dd-8fe4ee875edf service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:16:28.041490 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ced6542-e29a-41fc-a2dd-8fe4ee875edf service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:16:28.041770 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ced6542-e29a-41fc-a2dd-8fe4ee875edf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:16:28.043434 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-0ced6542-e29a-41fc-a2dd-8fe4ee875edf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:16:28.162854 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ced6542-e29a-41fc-a2dd-8fe4ee875edf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:16:28.164730 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ced6542-e29a-41fc-a2dd-8fe4ee875edf service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:16:28.185269 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 661/1314] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:16:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:16:30.735653 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b33002-e146-496d-aaf3-a27031ecb3fd None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:16:30.736240 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b33002-e146-496d-aaf3-a27031ecb3fd None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:16:30.736240 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b33002-e146-496d-aaf3-a27031ecb3fd None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:16:30.737635 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 654/1315] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:16:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:16:30.742842 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15113cbf-8eb7-4c90-aabb-47fbb93e6a86 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:16:30.743250 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15113cbf-8eb7-4c90-aabb-47fbb93e6a86 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:16:30.743397 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15113cbf-8eb7-4c90-aabb-47fbb93e6a86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:16:30.806970 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-15113cbf-8eb7-4c90-aabb-47fbb93e6a86 None None] MFA Rules not processed for user `e2f3ce08c4a046fcae882fe888ca766d`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:16:30.808236 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-15113cbf-8eb7-4c90-aabb-47fbb93e6a86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:16:30.866026 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:16:30.910998 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 662/1316] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:16:30 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:16:30.963995 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f45a3d-00c3-43cb-952e-f0fffadaff89 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:16:30.964283 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f45a3d-00c3-43cb-952e-f0fffadaff89 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:16:30.964562 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f45a3d-00c3-43cb-952e-f0fffadaff89 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:16:30.965959 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 655/1317] 10.222.0.44 () {54 vars in 964 bytes} [Wed Jun 29 11:16:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:16:30.971709 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2962765-0908-4d79-a246-42881e6b66ca None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:16:30.972030 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2962765-0908-4d79-a246-42881e6b66ca None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:16:30.972315 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2962765-0908-4d79-a246-42881e6b66ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:16:31.044956 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-c2962765-0908-4d79-a246-42881e6b66ca None None] MFA Rules not processed for user `b5f4bb9f989e4024a5da28dced646b92`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:16:31.046098 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-c2962765-0908-4d79-a246-42881e6b66ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:16:31.119855 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 663/1318] 10.222.0.44 () {58 vars in 1061 bytes} [Wed Jun 29 11:16:30 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:16:31.129383 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f45a3d-00c3-43cb-952e-f0fffadaff89 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:16:31.130895 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-b0f45a3d-00c3-43cb-952e-f0fffadaff89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:16:31.140531 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:16:31.231872 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f45a3d-00c3-43cb-952e-f0fffadaff89 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:16:31.302613 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57a4f02-9c14-4c37-b2f0-d125fc7a2e48 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f4bb9f989e4024a5da28dced646b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:16:31.303179 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57a4f02-9c14-4c37-b2f0-d125fc7a2e48 service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:16:31.303461 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57a4f02-9c14-4c37-b2f0-d125fc7a2e48 service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:16:31.303732 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57a4f02-9c14-4c37-b2f0-d125fc7a2e48 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:16:31.305391 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-e57a4f02-9c14-4c37-b2f0-d125fc7a2e48 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:16:31.364690 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:16:31.409149 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57a4f02-9c14-4c37-b2f0-d125fc7a2e48 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:16:31.410555 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57a4f02-9c14-4c37-b2f0-d125fc7a2e48 service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:16:31.428261 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 656/1319] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:16:31 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 11:16:34.632098 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d54514-16c8-4c42-878f-e9256686a5d9 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:16:34.632633 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d54514-16c8-4c42-878f-e9256686a5d9 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:16:34.632633 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d54514-16c8-4c42-878f-e9256686a5d9 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:16:34.634020 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 664/1320] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:16:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:16:34.639068 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6da745-e753-416f-a464-8fb3a06bafec None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:16:34.639450 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6da745-e753-416f-a464-8fb3a06bafec None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:16:34.639634 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6da745-e753-416f-a464-8fb3a06bafec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:16:34.702371 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-7d6da745-e753-416f-a464-8fb3a06bafec None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:16:34.704135 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-7d6da745-e753-416f-a464-8fb3a06bafec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:16:34.799078 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 657/1321] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:16:34 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:16:34.838019 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d54514-16c8-4c42-878f-e9256686a5d9 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:16:34.839620 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-96d54514-16c8-4c42-878f-e9256686a5d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:16:34.901385 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:16:34.954740 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d54514-16c8-4c42-878f-e9256686a5d9 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:16:35.026635 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac3f3a6-39e2-4145-8ea3-93ead543da23 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f4bb9f989e4024a5da28dced646b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:16:35.027281 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac3f3a6-39e2-4145-8ea3-93ead543da23 service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:16:35.027560 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac3f3a6-39e2-4145-8ea3-93ead543da23 service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:16:35.027850 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac3f3a6-39e2-4145-8ea3-93ead543da23 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:16:35.029495 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-4ac3f3a6-39e2-4145-8ea3-93ead543da23 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:16:35.122113 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:16:35.134869 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac3f3a6-39e2-4145-8ea3-93ead543da23 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:16:35.136330 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac3f3a6-39e2-4145-8ea3-93ead543da23 service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:16:35.154084 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 665/1322] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:16:34 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 11:16:38.184374 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46684aa2-edb4-40e5-beee-b8e20725c4dd None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:16:38.184887 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46684aa2-edb4-40e5-beee-b8e20725c4dd None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:16:38.184887 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46684aa2-edb4-40e5-beee-b8e20725c4dd None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:16:38.186244 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 658/1323] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:16:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:16:38.191490 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b7ef20-9203-4c24-8557-b412e2ba326a None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:16:38.191892 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b7ef20-9203-4c24-8557-b412e2ba326a None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:16:38.192195 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b7ef20-9203-4c24-8557-b412e2ba326a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:16:38.264567 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-84b7ef20-9203-4c24-8557-b412e2ba326a None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:16:38.265761 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-84b7ef20-9203-4c24-8557-b412e2ba326a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:16:38.364259 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 666/1324] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:16:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:16:38.400722 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46684aa2-edb4-40e5-beee-b8e20725c4dd None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:16:38.402378 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-46684aa2-edb4-40e5-beee-b8e20725c4dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:16:38.423148 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:16:38.519483 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46684aa2-edb4-40e5-beee-b8e20725c4dd None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:16:38.598173 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537e68a0-cc1d-4bab-81db-237e8bff57eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f4bb9f989e4024a5da28dced646b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:16:38.598835 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537e68a0-cc1d-4bab-81db-237e8bff57eb service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:16:38.599121 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537e68a0-cc1d-4bab-81db-237e8bff57eb service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:16:38.599396 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537e68a0-cc1d-4bab-81db-237e8bff57eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:16:38.601194 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-537e68a0-cc1d-4bab-81db-237e8bff57eb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:16:38.674624 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:16:38.712293 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537e68a0-cc1d-4bab-81db-237e8bff57eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:16:38.713730 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537e68a0-cc1d-4bab-81db-237e8bff57eb service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:16:38.731243 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 659/1325] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:16:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:16:40.845572 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3aae097-50ad-48c1-ae70-18e58d84a11e None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:16:40.846125 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3aae097-50ad-48c1-ae70-18e58d84a11e None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:16:40.846125 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3aae097-50ad-48c1-ae70-18e58d84a11e None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:16:40.847495 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 667/1326] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:16:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:16:40.852846 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd19bfc3-445a-44b1-b1e4-6734110c16e9 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:16:40.853294 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd19bfc3-445a-44b1-b1e4-6734110c16e9 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:16:40.853397 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd19bfc3-445a-44b1-b1e4-6734110c16e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:16:40.932593 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-bd19bfc3-445a-44b1-b1e4-6734110c16e9 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:16:40.933726 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-bd19bfc3-445a-44b1-b1e4-6734110c16e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:16:41.029990 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 660/1327] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:16:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:16:41.065475 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3aae097-50ad-48c1-ae70-18e58d84a11e None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:16:41.067085 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-c3aae097-50ad-48c1-ae70-18e58d84a11e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:16:41.127395 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:16:41.170873 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3aae097-50ad-48c1-ae70-18e58d84a11e None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:16:41.249220 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56007bb1-63f9-4f98-a9a6-6d0ed6f94d5d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f4bb9f989e4024a5da28dced646b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:16:41.249866 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56007bb1-63f9-4f98-a9a6-6d0ed6f94d5d service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:16:41.250148 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56007bb1-63f9-4f98-a9a6-6d0ed6f94d5d service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:16:41.250937 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56007bb1-63f9-4f98-a9a6-6d0ed6f94d5d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:16:41.252654 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-56007bb1-63f9-4f98-a9a6-6d0ed6f94d5d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:16:41.362533 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:16:41.365221 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56007bb1-63f9-4f98-a9a6-6d0ed6f94d5d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:16:41.366727 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56007bb1-63f9-4f98-a9a6-6d0ed6f94d5d service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:16:41.386623 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 668/1328] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:16:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:16:43.454433 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344648bb-7329-4fc5-92cd-9b33b6b97239 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:16:43.454981 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344648bb-7329-4fc5-92cd-9b33b6b97239 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:16:43.454981 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344648bb-7329-4fc5-92cd-9b33b6b97239 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:16:43.456414 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 661/1329] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:16:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:16:43.462177 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2990fca-4a40-402c-8b79-58ff6769885a None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:16:43.462664 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2990fca-4a40-402c-8b79-58ff6769885a None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:16:43.462756 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2990fca-4a40-402c-8b79-58ff6769885a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:16:43.531158 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-a2990fca-4a40-402c-8b79-58ff6769885a None None] MFA Rules not processed for user `e2f3ce08c4a046fcae882fe888ca766d`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:16:43.532951 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-a2990fca-4a40-402c-8b79-58ff6769885a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:16:43.628380 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 669/1330] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:16:43 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:16:43.665839 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344648bb-7329-4fc5-92cd-9b33b6b97239 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:16:43.667529 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-344648bb-7329-4fc5-92cd-9b33b6b97239 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:16:43.701337 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:16:43.777725 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344648bb-7329-4fc5-92cd-9b33b6b97239 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:16:43.852928 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18144f13-03ff-455e-8069-d18e303de8e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f4bb9f989e4024a5da28dced646b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:16:43.853580 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18144f13-03ff-455e-8069-d18e303de8e4 service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:16:43.853861 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18144f13-03ff-455e-8069-d18e303de8e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:16:43.854136 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18144f13-03ff-455e-8069-d18e303de8e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:16:43.855764 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-18144f13-03ff-455e-8069-d18e303de8e4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:16:43.933959 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:16:43.966424 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18144f13-03ff-455e-8069-d18e303de8e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:16:43.968000 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18144f13-03ff-455e-8069-d18e303de8e4 service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:16:43.987812 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 662/1331] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:16:43 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:16:46.510708 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce78d30c-2d35-43f4-a55c-346de1223f38 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:16:46.511309 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce78d30c-2d35-43f4-a55c-346de1223f38 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:16:46.511309 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce78d30c-2d35-43f4-a55c-346de1223f38 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:16:46.512642 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 670/1332] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:16:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:16:46.517833 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56381f15-2059-410b-a5f6-e09c35469891 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:16:46.518247 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56381f15-2059-410b-a5f6-e09c35469891 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:16:46.518391 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56381f15-2059-410b-a5f6-e09c35469891 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:16:46.581630 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-56381f15-2059-410b-a5f6-e09c35469891 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:16:46.582829 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-56381f15-2059-410b-a5f6-e09c35469891 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:16:46.676779 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 663/1333] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:16:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:16:46.712053 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce78d30c-2d35-43f4-a55c-346de1223f38 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:16:46.713580 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-ce78d30c-2d35-43f4-a55c-346de1223f38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:16:46.781196 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:16:46.814781 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce78d30c-2d35-43f4-a55c-346de1223f38 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:16:46.886743 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec0c4cb-1343-4a72-9ed0-3a807b73143a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f4bb9f989e4024a5da28dced646b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:16:46.887385 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec0c4cb-1343-4a72-9ed0-3a807b73143a service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:16:46.887668 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec0c4cb-1343-4a72-9ed0-3a807b73143a service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:16:46.887981 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec0c4cb-1343-4a72-9ed0-3a807b73143a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:16:46.889658 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-fec0c4cb-1343-4a72-9ed0-3a807b73143a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:16:47.002308 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec0c4cb-1343-4a72-9ed0-3a807b73143a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:16:47.003768 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec0c4cb-1343-4a72-9ed0-3a807b73143a service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:16:47.013481 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:16:47.022074 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 671/1334] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:16:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:16:49.131230 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52db3164-1f56-43ca-9b7d-86e89be044a7 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:16:49.131788 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52db3164-1f56-43ca-9b7d-86e89be044a7 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:16:49.131788 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52db3164-1f56-43ca-9b7d-86e89be044a7 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:16:49.133205 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 664/1335] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:16:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:16:49.138493 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397ca047-7037-47c1-8c29-3b21c90294c1 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:16:49.138933 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397ca047-7037-47c1-8c29-3b21c90294c1 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:16:49.139046 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397ca047-7037-47c1-8c29-3b21c90294c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:16:49.202798 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-397ca047-7037-47c1-8c29-3b21c90294c1 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:16:49.204039 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-397ca047-7037-47c1-8c29-3b21c90294c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:16:49.296550 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 672/1336] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:16:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:16:49.336427 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52db3164-1f56-43ca-9b7d-86e89be044a7 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:16:49.337967 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-52db3164-1f56-43ca-9b7d-86e89be044a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:16:49.378031 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:16:49.444745 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52db3164-1f56-43ca-9b7d-86e89be044a7 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:16:49.516686 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41289d1-d67e-4c44-9806-c6b24387a5ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f4bb9f989e4024a5da28dced646b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:16:49.517281 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41289d1-d67e-4c44-9806-c6b24387a5ce service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:16:49.517568 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41289d1-d67e-4c44-9806-c6b24387a5ce service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:16:49.517849 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41289d1-d67e-4c44-9806-c6b24387a5ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:16:49.519512 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-b41289d1-d67e-4c44-9806-c6b24387a5ce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:16:49.602760 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:16:49.628276 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41289d1-d67e-4c44-9806-c6b24387a5ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:16:49.629737 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41289d1-d67e-4c44-9806-c6b24387a5ce service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:16:49.647389 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 665/1337] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:16:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:16:52.413736 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136f7d6b-3894-4df4-8194-492ff25e9131 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:16:52.414268 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136f7d6b-3894-4df4-8194-492ff25e9131 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:16:52.414385 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136f7d6b-3894-4df4-8194-492ff25e9131 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:16:52.415654 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 673/1338] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:16:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:16:52.420934 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739ff212-22c0-4371-b7e8-c98fe102e045 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:16:52.421313 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739ff212-22c0-4371-b7e8-c98fe102e045 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:16:52.421492 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739ff212-22c0-4371-b7e8-c98fe102e045 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:16:52.486499 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-739ff212-22c0-4371-b7e8-c98fe102e045 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:16:52.487685 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-739ff212-22c0-4371-b7e8-c98fe102e045 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:16:52.587534 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 666/1339] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:16:52 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:16:52.623978 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136f7d6b-3894-4df4-8194-492ff25e9131 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:16:52.625540 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-136f7d6b-3894-4df4-8194-492ff25e9131 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:16:52.699124 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:16:52.727102 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136f7d6b-3894-4df4-8194-492ff25e9131 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:16:52.799267 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1234c131-36f6-43a6-81a8-8059908bc66e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f4bb9f989e4024a5da28dced646b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:16:52.799940 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1234c131-36f6-43a6-81a8-8059908bc66e service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:16:52.800231 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1234c131-36f6-43a6-81a8-8059908bc66e service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:16:52.800510 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1234c131-36f6-43a6-81a8-8059908bc66e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:16:52.802153 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-1234c131-36f6-43a6-81a8-8059908bc66e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:16:52.908363 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1234c131-36f6-43a6-81a8-8059908bc66e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:16:52.909798 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1234c131-36f6-43a6-81a8-8059908bc66e service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:16:52.927841 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 674/1340] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:16:52 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:16:57.216821 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322824a4-de66-4a47-93dc-f1c119c9fbb2 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:16:57.217371 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322824a4-de66-4a47-93dc-f1c119c9fbb2 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:16:57.217371 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322824a4-de66-4a47-93dc-f1c119c9fbb2 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:16:57.218725 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 667/1341] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:16:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:16:57.225351 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657fe619-480f-4ee1-8a2e-ef6e5732301b None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:16:57.226080 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657fe619-480f-4ee1-8a2e-ef6e5732301b None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:16:57.226367 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657fe619-480f-4ee1-8a2e-ef6e5732301b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:16:57.233559 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:16:57.305409 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-657fe619-480f-4ee1-8a2e-ef6e5732301b None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:16:57.305560 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-657fe619-480f-4ee1-8a2e-ef6e5732301b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:16:57.423515 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 675/1342] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:16:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:16:57.463031 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322824a4-de66-4a47-93dc-f1c119c9fbb2 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:16:57.465235 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-322824a4-de66-4a47-93dc-f1c119c9fbb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:16:57.515379 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:16:57.560946 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322824a4-de66-4a47-93dc-f1c119c9fbb2 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:16:57.565241 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba337c7-966f-4326-8a0a-b6a0bd9763bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f4bb9f989e4024a5da28dced646b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:16:57.565839 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba337c7-966f-4326-8a0a-b6a0bd9763bd service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:16:57.566126 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba337c7-966f-4326-8a0a-b6a0bd9763bd service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:16:57.566406 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba337c7-966f-4326-8a0a-b6a0bd9763bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:16:57.575892 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba337c7-966f-4326-8a0a-b6a0bd9763bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:16:57.577316 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba337c7-966f-4326-8a0a-b6a0bd9763bd service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:16:57.587045 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 668/1343] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:16:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:17:00.659974 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d66c48-325a-42bf-af3b-393f01c6cbd6 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:17:00.660519 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d66c48-325a-42bf-af3b-393f01c6cbd6 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:17:00.660519 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d66c48-325a-42bf-af3b-393f01c6cbd6 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:17:00.661925 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 676/1344] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:17:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:17:00.667750 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3e4168-0634-4339-bd87-b65c55d8256f None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:17:00.668288 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3e4168-0634-4339-bd87-b65c55d8256f None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:17:00.668491 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3e4168-0634-4339-bd87-b65c55d8256f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:17:00.696348 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-4c3e4168-0634-4339-bd87-b65c55d8256f None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:17:00.697554 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-4c3e4168-0634-4339-bd87-b65c55d8256f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:17:00.777429 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 669/1345] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:17:00 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:17:00.815315 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d66c48-325a-42bf-af3b-393f01c6cbd6 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:17:00.826228 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d66c48-325a-42bf-af3b-393f01c6cbd6 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:17:00.830370 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e25b60b-2849-4aab-a180-9b4de18b8ff3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f4bb9f989e4024a5da28dced646b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:17:00.831006 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e25b60b-2849-4aab-a180-9b4de18b8ff3 service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:17:00.831466 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e25b60b-2849-4aab-a180-9b4de18b8ff3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:17:00.831747 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e25b60b-2849-4aab-a180-9b4de18b8ff3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:17:00.841162 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e25b60b-2849-4aab-a180-9b4de18b8ff3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:17:00.842631 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e25b60b-2849-4aab-a180-9b4de18b8ff3 service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:17:00.852454 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 677/1346] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:17:00 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:17:05.016387 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909f6f09-e0a1-467a-abb3-9d3eb425e219 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:17:05.016949 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909f6f09-e0a1-467a-abb3-9d3eb425e219 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:17:05.016949 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909f6f09-e0a1-467a-abb3-9d3eb425e219 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:17:05.018316 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 670/1347] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:17:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:17:05.023522 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0132db-a593-4524-926d-70fde75d922e None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:17:05.024031 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0132db-a593-4524-926d-70fde75d922e None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:17:05.024131 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0132db-a593-4524-926d-70fde75d922e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:17:05.041373 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-cc0132db-a593-4524-926d-70fde75d922e None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:17:05.047655 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-cc0132db-a593-4524-926d-70fde75d922e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:17:05.050897 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 678/1348] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:17:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:17:05.089033 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909f6f09-e0a1-467a-abb3-9d3eb425e219 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:17:05.103570 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909f6f09-e0a1-467a-abb3-9d3eb425e219 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:17:05.107218 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d658064-96a3-4bb5-b4f2-f195e13c43dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f4bb9f989e4024a5da28dced646b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:17:05.107891 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d658064-96a3-4bb5-b4f2-f195e13c43dc service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:17:05.108179 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d658064-96a3-4bb5-b4f2-f195e13c43dc service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:17:05.108612 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d658064-96a3-4bb5-b4f2-f195e13c43dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:17:05.122052 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d658064-96a3-4bb5-b4f2-f195e13c43dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:17:05.123596 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d658064-96a3-4bb5-b4f2-f195e13c43dc service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:17:05.134715 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 671/1349] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:17:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:17:10.108386 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dfad7d-cf28-4e37-82b4-b7db94788f10 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:17:10.108902 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dfad7d-cf28-4e37-82b4-b7db94788f10 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:17:10.109019 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dfad7d-cf28-4e37-82b4-b7db94788f10 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:17:10.111111 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 679/1350] 10.222.0.44 () {54 vars in 921 bytes} [Wed Jun 29 11:17:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:17:10.115717 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b744620-93d9-4cde-b691-7d93d7819ffe None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:17:10.116147 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b744620-93d9-4cde-b691-7d93d7819ffe None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:17:10.116306 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b744620-93d9-4cde-b691-7d93d7819ffe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:17:10.215796 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-5b744620-93d9-4cde-b691-7d93d7819ffe None None] MFA Rules not processed for user `1a0505342b704d02977a5125957c2c9e`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:17:10.217266 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-5b744620-93d9-4cde-b691-7d93d7819ffe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:17:10.248675 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:17:10.303421 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 672/1351] 10.222.0.44 () {58 vars in 1018 bytes} [Wed Jun 29 11:17:10 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:17:14.601596 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cd50df-d3d5-4099-b1a2-3a8f1371d10e None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:17:14.602123 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cd50df-d3d5-4099-b1a2-3a8f1371d10e None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:17:14.602203 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cd50df-d3d5-4099-b1a2-3a8f1371d10e None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:17:14.603798 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 680/1352] 10.222.0.44 () {54 vars in 921 bytes} [Wed Jun 29 11:17:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:17:14.609490 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ebe694-ba97-4d26-9611-c25fa4c7770b None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:17:14.610537 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ebe694-ba97-4d26-9611-c25fa4c7770b None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:17:14.610537 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ebe694-ba97-4d26-9611-c25fa4c7770b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:17:14.629306 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-56ebe694-ba97-4d26-9611-c25fa4c7770b None None] MFA Rules not processed for user `1a0505342b704d02977a5125957c2c9e`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:17:14.630510 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-56ebe694-ba97-4d26-9611-c25fa4c7770b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:17:14.640095 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 673/1353] 10.222.0.44 () {58 vars in 1018 bytes} [Wed Jun 29 11:17:14 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:17:15.037439 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=72 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:17:15.973571 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd0f4a3-baaa-48bd-bfd4-f4850fb12169 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:17:15.974264 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd0f4a3-baaa-48bd-bfd4-f4850fb12169 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:17:15.974264 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd0f4a3-baaa-48bd-bfd4-f4850fb12169 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:17:15.975468 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 681/1354] 10.222.0.44 () {54 vars in 921 bytes} [Wed Jun 29 11:17:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:17:15.980630 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8750a438-302c-4a0f-981d-99c46279de5a None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:17:15.981103 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8750a438-302c-4a0f-981d-99c46279de5a None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:17:15.981247 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8750a438-302c-4a0f-981d-99c46279de5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:17:16.000169 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-8750a438-302c-4a0f-981d-99c46279de5a None None] MFA Rules not processed for user `1a0505342b704d02977a5125957c2c9e`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:17:16.001363 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-8750a438-302c-4a0f-981d-99c46279de5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:17:16.009840 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 674/1355] 10.222.0.44 () {58 vars in 1018 bytes} [Wed Jun 29 11:17:15 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:17:19.613232 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3792cf6d-ee73-4288-aff3-e8661a3323bf None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:17:19.613761 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3792cf6d-ee73-4288-aff3-e8661a3323bf None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:17:19.613761 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3792cf6d-ee73-4288-aff3-e8661a3323bf None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:17:19.617413 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 682/1356] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:17:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:17:19.621570 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1633bdd4-b539-4585-934f-a5ccef2cbe62 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:17:19.622463 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1633bdd4-b539-4585-934f-a5ccef2cbe62 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:17:19.622749 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1633bdd4-b539-4585-934f-a5ccef2cbe62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:17:19.646238 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-1633bdd4-b539-4585-934f-a5ccef2cbe62 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:17:19.647435 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-1633bdd4-b539-4585-934f-a5ccef2cbe62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:17:19.658333 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 675/1357] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:17:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:17:19.715712 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca78665-e80a-4e95-966c-6e33399b7bbc None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:17:19.716055 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca78665-e80a-4e95-966c-6e33399b7bbc None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:17:19.716291 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca78665-e80a-4e95-966c-6e33399b7bbc None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:17:19.717787 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 683/1358] 10.222.0.44 () {54 vars in 961 bytes} [Wed Jun 29 11:17:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:17:19.726395 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a385fe-14d0-4e0b-b7b3-102b605c4d70 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:17:19.727977 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a385fe-14d0-4e0b-b7b3-102b605c4d70 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:17:19.728478 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a385fe-14d0-4e0b-b7b3-102b605c4d70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:17:19.788756 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-97a385fe-14d0-4e0b-b7b3-102b605c4d70 None None] MFA Rules not processed for user `796aeeff16df4739bddec1f6dee61bfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:17:19.789872 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-97a385fe-14d0-4e0b-b7b3-102b605c4d70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:17:19.878145 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 676/1359] 10.222.0.44 () {58 vars in 1058 bytes} [Wed Jun 29 11:17:19 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:17:19.886375 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca78665-e80a-4e95-966c-6e33399b7bbc None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:17:19.902831 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca78665-e80a-4e95-966c-6e33399b7bbc None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:17:19.909164 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2701407-7c65-4b62-b367-b61868ab6d2a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '796aeeff16df4739bddec1f6dee61bfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:17:19.910050 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2701407-7c65-4b62-b367-b61868ab6d2a service nova] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:17:19.910485 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2701407-7c65-4b62-b367-b61868ab6d2a service nova] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:17:19.910931 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2701407-7c65-4b62-b367-b61868ab6d2a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:17:19.921010 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2701407-7c65-4b62-b367-b61868ab6d2a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:17:19.923154 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2701407-7c65-4b62-b367-b61868ab6d2a service nova] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:17:19.933217 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 684/1360] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:17:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:17:22.200662 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32c547c-2baa-42a2-88e9-77d30ee29258 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:17:22.201133 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32c547c-2baa-42a2-88e9-77d30ee29258 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:17:22.201404 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32c547c-2baa-42a2-88e9-77d30ee29258 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:17:22.202891 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 677/1361] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:17:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:17:22.208136 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67174c6-e190-43d1-8e9f-94daa1deea78 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:17:22.208585 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67174c6-e190-43d1-8e9f-94daa1deea78 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:17:22.208895 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67174c6-e190-43d1-8e9f-94daa1deea78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:17:22.227954 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-a67174c6-e190-43d1-8e9f-94daa1deea78 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:17:22.229241 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-a67174c6-e190-43d1-8e9f-94daa1deea78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:17:22.238643 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 685/1362] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:17:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:17:22.282259 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32c547c-2baa-42a2-88e9-77d30ee29258 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:17:22.297958 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32c547c-2baa-42a2-88e9-77d30ee29258 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:17:22.303337 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acd144c-b490-4a23-87f8-c48eac9eb158 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '796aeeff16df4739bddec1f6dee61bfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:17:22.304006 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acd144c-b490-4a23-87f8-c48eac9eb158 service nova] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:17:22.304296 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acd144c-b490-4a23-87f8-c48eac9eb158 service nova] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:17:22.304571 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acd144c-b490-4a23-87f8-c48eac9eb158 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:17:22.313642 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acd144c-b490-4a23-87f8-c48eac9eb158 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:17:22.315866 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acd144c-b490-4a23-87f8-c48eac9eb158 service nova] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:17:22.326291 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 678/1363] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:17:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 11:17:24.790640 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b320ea03-197d-4c6a-bf74-be65a40f25d7 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:17:24.791212 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b320ea03-197d-4c6a-bf74-be65a40f25d7 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:17:24.791212 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b320ea03-197d-4c6a-bf74-be65a40f25d7 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:17:24.792566 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 686/1364] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:17:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:17:24.798797 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9aa6551-5d5a-424a-8208-727e044ce06b None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:17:24.799694 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9aa6551-5d5a-424a-8208-727e044ce06b None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:17:24.800212 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9aa6551-5d5a-424a-8208-727e044ce06b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:17:24.823077 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-f9aa6551-5d5a-424a-8208-727e044ce06b None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:17:24.824669 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-f9aa6551-5d5a-424a-8208-727e044ce06b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:17:24.832971 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 679/1365] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:17:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:17:24.879818 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa131d10-7453-4ced-bb12-e932986d63b3 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:17:24.880166 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa131d10-7453-4ced-bb12-e932986d63b3 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:17:24.880450 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa131d10-7453-4ced-bb12-e932986d63b3 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:17:24.881792 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 687/1366] 10.222.0.44 () {54 vars in 961 bytes} [Wed Jun 29 11:17:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:17:24.887193 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41720854-bcf1-4118-807f-e90fc1ba98be None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:17:24.887610 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41720854-bcf1-4118-807f-e90fc1ba98be None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:17:24.888043 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41720854-bcf1-4118-807f-e90fc1ba98be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:17:24.906074 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-41720854-bcf1-4118-807f-e90fc1ba98be None None] MFA Rules not processed for user `796aeeff16df4739bddec1f6dee61bfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:17:24.907737 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-41720854-bcf1-4118-807f-e90fc1ba98be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:17:24.920036 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 680/1367] 10.222.0.44 () {58 vars in 1058 bytes} [Wed Jun 29 11:17:24 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:17:24.928199 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa131d10-7453-4ced-bb12-e932986d63b3 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:17:24.942443 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa131d10-7453-4ced-bb12-e932986d63b3 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:17:24.947697 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f7d2df-3e86-4d28-bef2-13a3c3918d58 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '796aeeff16df4739bddec1f6dee61bfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:17:24.948357 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f7d2df-3e86-4d28-bef2-13a3c3918d58 service nova] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:17:24.948639 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f7d2df-3e86-4d28-bef2-13a3c3918d58 service nova] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:17:24.948915 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f7d2df-3e86-4d28-bef2-13a3c3918d58 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:17:24.957146 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f7d2df-3e86-4d28-bef2-13a3c3918d58 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:17:24.958550 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f7d2df-3e86-4d28-bef2-13a3c3918d58 service nova] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:17:24.968203 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 688/1368] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:17:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:17:27.288313 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018ab05e-26c2-4eb2-91a5-e3da1cd5e5ae None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:17:27.288961 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018ab05e-26c2-4eb2-91a5-e3da1cd5e5ae None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:17:27.288961 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018ab05e-26c2-4eb2-91a5-e3da1cd5e5ae None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:17:27.290234 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 681/1369] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:17:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:17:27.295438 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7149c14-0b4e-4183-9c35-862f5f32ee95 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:17:27.295874 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7149c14-0b4e-4183-9c35-862f5f32ee95 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:17:27.296032 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7149c14-0b4e-4183-9c35-862f5f32ee95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:17:27.314886 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-e7149c14-0b4e-4183-9c35-862f5f32ee95 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:17:27.316351 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-e7149c14-0b4e-4183-9c35-862f5f32ee95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:17:27.331881 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 689/1370] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:17:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:17:27.378705 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018ab05e-26c2-4eb2-91a5-e3da1cd5e5ae None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:17:27.393576 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018ab05e-26c2-4eb2-91a5-e3da1cd5e5ae None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:17:27.398950 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f17f12-3d82-4aaf-953e-d2a51d49bdff service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '796aeeff16df4739bddec1f6dee61bfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:17:27.399588 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f17f12-3d82-4aaf-953e-d2a51d49bdff service nova] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:17:27.399901 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f17f12-3d82-4aaf-953e-d2a51d49bdff service nova] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:17:27.400184 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f17f12-3d82-4aaf-953e-d2a51d49bdff service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:17:27.408397 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f17f12-3d82-4aaf-953e-d2a51d49bdff service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:17:27.409971 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f17f12-3d82-4aaf-953e-d2a51d49bdff service nova] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:17:27.419668 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 682/1371] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:17:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:17:27.898015 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afe750c-292a-467d-b081-fa0dd1e42fb7 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:17:27.898420 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afe750c-292a-467d-b081-fa0dd1e42fb7 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:17:27.898820 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afe750c-292a-467d-b081-fa0dd1e42fb7 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:17:27.900375 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 690/1372] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:17:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:17:27.905564 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a367d167-7563-4f62-87be-48456d50fa5e None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:17:27.905863 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a367d167-7563-4f62-87be-48456d50fa5e None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:17:27.906167 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a367d167-7563-4f62-87be-48456d50fa5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:17:27.932980 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-a367d167-7563-4f62-87be-48456d50fa5e None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:17:27.934194 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-a367d167-7563-4f62-87be-48456d50fa5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:17:27.943876 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 683/1373] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:17:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:17:28.300142 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb94dba-b0f9-42db-b4ec-53c7cffe7f18 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:17:28.300622 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb94dba-b0f9-42db-b4ec-53c7cffe7f18 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:17:28.300925 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb94dba-b0f9-42db-b4ec-53c7cffe7f18 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:17:28.302528 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 691/1374] 10.222.0.44 () {54 vars in 963 bytes} [Wed Jun 29 11:17:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:17:28.307391 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b36313-c816-46fe-8bf4-0adeb8f39b0c None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:17:28.307863 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b36313-c816-46fe-8bf4-0adeb8f39b0c None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:17:28.308189 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b36313-c816-46fe-8bf4-0adeb8f39b0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:17:28.315452 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:17:28.358508 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-97b36313-c816-46fe-8bf4-0adeb8f39b0c None None] MFA Rules not processed for user `ff7931b75a924394b1af733febce0834`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:17:28.360427 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-97b36313-c816-46fe-8bf4-0adeb8f39b0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:17:28.423456 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 684/1375] 10.222.0.44 () {58 vars in 1060 bytes} [Wed Jun 29 11:17:28 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:17:28.430914 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb94dba-b0f9-42db-b4ec-53c7cffe7f18 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:17:28.442204 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb94dba-b0f9-42db-b4ec-53c7cffe7f18 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:17:28.445816 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc4e8de-8ee8-42a4-b4c4-1ae73620accb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff7931b75a924394b1af733febce0834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:17:28.446557 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc4e8de-8ee8-42a4-b4c4-1ae73620accb service cinder] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:17:28.447057 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc4e8de-8ee8-42a4-b4c4-1ae73620accb service cinder] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:17:28.447406 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc4e8de-8ee8-42a4-b4c4-1ae73620accb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:17:28.457114 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc4e8de-8ee8-42a4-b4c4-1ae73620accb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:17:28.458614 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc4e8de-8ee8-42a4-b4c4-1ae73620accb service cinder] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:17:28.468392 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 692/1376] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:17:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:17:29.607298 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df05365-ba72-446a-a5ed-115339d8e288 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:17:29.607825 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df05365-ba72-446a-a5ed-115339d8e288 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:17:29.607825 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df05365-ba72-446a-a5ed-115339d8e288 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:17:29.610297 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 685/1377] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:17:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:17:29.615008 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0310a15-04c8-4e22-bf5d-24e49d6adced None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:17:29.615418 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0310a15-04c8-4e22-bf5d-24e49d6adced None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:17:29.615558 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0310a15-04c8-4e22-bf5d-24e49d6adced None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:17:29.635073 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-b0310a15-04c8-4e22-bf5d-24e49d6adced None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:17:29.636655 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-b0310a15-04c8-4e22-bf5d-24e49d6adced None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:17:29.645942 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 693/1378] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:17:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:17:29.690891 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df05365-ba72-446a-a5ed-115339d8e288 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:17:29.706825 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df05365-ba72-446a-a5ed-115339d8e288 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:17:29.713246 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae74a55-9f61-4d70-b224-1d45c819f9c2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '796aeeff16df4739bddec1f6dee61bfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:17:29.714130 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae74a55-9f61-4d70-b224-1d45c819f9c2 service nova] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:17:29.714608 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae74a55-9f61-4d70-b224-1d45c819f9c2 service nova] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:17:29.715059 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae74a55-9f61-4d70-b224-1d45c819f9c2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:17:29.726568 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae74a55-9f61-4d70-b224-1d45c819f9c2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:17:29.728387 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae74a55-9f61-4d70-b224-1d45c819f9c2 service nova] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:17:29.740001 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 686/1379] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:17:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:17:31.899459 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8325e9fa-88fe-4c62-b351-c95153896352 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:17:31.900031 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8325e9fa-88fe-4c62-b351-c95153896352 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:17:31.900031 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8325e9fa-88fe-4c62-b351-c95153896352 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:17:31.901404 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 694/1380] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:17:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:17:31.907241 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82720a1c-6105-4c8a-88ee-394c0926635a None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:17:31.907679 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82720a1c-6105-4c8a-88ee-394c0926635a None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:17:31.907788 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82720a1c-6105-4c8a-88ee-394c0926635a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:17:31.930296 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-82720a1c-6105-4c8a-88ee-394c0926635a None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:17:31.931521 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-82720a1c-6105-4c8a-88ee-394c0926635a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:17:31.939205 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 687/1381] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:17:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:17:31.985944 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8325e9fa-88fe-4c62-b351-c95153896352 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:17:32.003157 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8325e9fa-88fe-4c62-b351-c95153896352 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:17:32.011004 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7901b27c-fb42-4643-96c1-fb82dd0e1d88 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '796aeeff16df4739bddec1f6dee61bfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:17:32.011647 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7901b27c-fb42-4643-96c1-fb82dd0e1d88 service nova] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:17:32.011983 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7901b27c-fb42-4643-96c1-fb82dd0e1d88 service nova] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:17:32.012255 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7901b27c-fb42-4643-96c1-fb82dd0e1d88 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:17:32.022118 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7901b27c-fb42-4643-96c1-fb82dd0e1d88 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:17:32.023700 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7901b27c-fb42-4643-96c1-fb82dd0e1d88 service nova] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:17:32.034179 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 695/1382] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:17:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:17:34.211908 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1725d4c-d9b8-45df-b62d-6b93eba8fe66 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:17:34.212664 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1725d4c-d9b8-45df-b62d-6b93eba8fe66 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:17:34.212664 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1725d4c-d9b8-45df-b62d-6b93eba8fe66 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:17:34.214008 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 688/1383] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:17:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:17:34.219409 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7033dd-7610-4d2c-acd1-2eb3f6294e8d None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:17:34.220041 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7033dd-7610-4d2c-acd1-2eb3f6294e8d None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:17:34.220041 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7033dd-7610-4d2c-acd1-2eb3f6294e8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:17:34.238377 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-2b7033dd-7610-4d2c-acd1-2eb3f6294e8d None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:17:34.239578 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-2b7033dd-7610-4d2c-acd1-2eb3f6294e8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:17:34.247517 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 696/1384] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:17:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:17:34.288915 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1725d4c-d9b8-45df-b62d-6b93eba8fe66 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:17:34.304125 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1725d4c-d9b8-45df-b62d-6b93eba8fe66 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:17:34.309693 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb71573-b64b-4f26-863a-3c90f078c88a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '796aeeff16df4739bddec1f6dee61bfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:17:34.310450 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb71573-b64b-4f26-863a-3c90f078c88a service nova] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:17:34.310826 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb71573-b64b-4f26-863a-3c90f078c88a service nova] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:17:34.311221 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb71573-b64b-4f26-863a-3c90f078c88a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:17:34.320384 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb71573-b64b-4f26-863a-3c90f078c88a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:17:34.322053 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb71573-b64b-4f26-863a-3c90f078c88a service nova] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:17:34.332418 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 689/1385] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:17:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:17:36.461611 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45345cdd-5195-41cc-a3b7-61e5acb10a01 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:17:36.462433 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45345cdd-5195-41cc-a3b7-61e5acb10a01 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:17:36.462743 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45345cdd-5195-41cc-a3b7-61e5acb10a01 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:17:36.464569 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 697/1386] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:17:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:17:36.471197 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b814d84f-f381-43a5-aaaf-2239f30d8a3c None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:17:36.471642 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b814d84f-f381-43a5-aaaf-2239f30d8a3c None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:17:36.471766 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b814d84f-f381-43a5-aaaf-2239f30d8a3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:17:36.490450 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-b814d84f-f381-43a5-aaaf-2239f30d8a3c None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:17:36.491667 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-b814d84f-f381-43a5-aaaf-2239f30d8a3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:17:36.499669 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 690/1387] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:17:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:17:36.545972 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45345cdd-5195-41cc-a3b7-61e5acb10a01 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:17:36.560953 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45345cdd-5195-41cc-a3b7-61e5acb10a01 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:17:36.567352 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54582733-238c-4856-9461-261df3606b62 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '796aeeff16df4739bddec1f6dee61bfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:17:36.568130 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54582733-238c-4856-9461-261df3606b62 service nova] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:17:36.568447 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54582733-238c-4856-9461-261df3606b62 service nova] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:17:36.568764 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54582733-238c-4856-9461-261df3606b62 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:17:36.577259 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54582733-238c-4856-9461-261df3606b62 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:17:36.579010 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54582733-238c-4856-9461-261df3606b62 service nova] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:17:36.589003 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 698/1388] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:17:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:17:38.779708 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab73044c-e4de-41b4-96f2-937d2b9a5527 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:17:38.780694 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab73044c-e4de-41b4-96f2-937d2b9a5527 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:17:38.780694 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab73044c-e4de-41b4-96f2-937d2b9a5527 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:17:38.781663 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 691/1389] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:17:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:17:38.787042 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf9cbb9-313a-4127-b2d9-58d54c7e3826 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:17:38.787479 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf9cbb9-313a-4127-b2d9-58d54c7e3826 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:17:38.787592 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf9cbb9-313a-4127-b2d9-58d54c7e3826 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:17:38.805398 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-bdf9cbb9-313a-4127-b2d9-58d54c7e3826 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:17:38.806591 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-bdf9cbb9-313a-4127-b2d9-58d54c7e3826 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:17:38.815202 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 699/1390] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:17:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 11:17:38.858351 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab73044c-e4de-41b4-96f2-937d2b9a5527 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:17:38.873151 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab73044c-e4de-41b4-96f2-937d2b9a5527 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:17:38.878743 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f726aa-e203-458f-8595-9b0c150f242f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '796aeeff16df4739bddec1f6dee61bfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:17:38.879377 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f726aa-e203-458f-8595-9b0c150f242f service nova] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:17:38.879660 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f726aa-e203-458f-8595-9b0c150f242f service nova] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:17:38.879975 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f726aa-e203-458f-8595-9b0c150f242f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:17:38.888456 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f726aa-e203-458f-8595-9b0c150f242f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:17:38.890016 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f726aa-e203-458f-8595-9b0c150f242f service nova] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:17:38.899887 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 692/1391] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:17:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 11:17:41.165707 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d76b8e-cd8a-4440-8259-40efa9ae4432 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:17:41.166233 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d76b8e-cd8a-4440-8259-40efa9ae4432 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:17:41.166233 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d76b8e-cd8a-4440-8259-40efa9ae4432 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:17:41.167614 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 700/1392] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:17:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:17:41.176092 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020777b7-24dd-4596-9920-745ea37c03f0 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:17:41.176513 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020777b7-24dd-4596-9920-745ea37c03f0 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:17:41.176604 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020777b7-24dd-4596-9920-745ea37c03f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:17:41.194032 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-020777b7-24dd-4596-9920-745ea37c03f0 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:17:41.195232 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-020777b7-24dd-4596-9920-745ea37c03f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:17:41.203762 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 693/1393] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:17:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:17:41.247792 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d76b8e-cd8a-4440-8259-40efa9ae4432 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:17:41.262610 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d76b8e-cd8a-4440-8259-40efa9ae4432 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:17:41.268704 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6239fa-39c0-4168-b0f9-b24b080f86b9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '796aeeff16df4739bddec1f6dee61bfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:17:41.269345 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6239fa-39c0-4168-b0f9-b24b080f86b9 service nova] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:17:41.269629 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6239fa-39c0-4168-b0f9-b24b080f86b9 service nova] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:17:41.269909 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6239fa-39c0-4168-b0f9-b24b080f86b9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:17:41.278407 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6239fa-39c0-4168-b0f9-b24b080f86b9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:17:41.280001 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6239fa-39c0-4168-b0f9-b24b080f86b9 service nova] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:17:41.289615 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 701/1394] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:17:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 11:17:43.516566 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756fcda2-963e-4188-8b36-ef40309b1c02 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:17:43.517130 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756fcda2-963e-4188-8b36-ef40309b1c02 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:17:43.517130 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756fcda2-963e-4188-8b36-ef40309b1c02 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:17:43.518544 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 694/1395] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:17:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:17:43.523922 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e950dbf8-4710-4aea-9282-742957103d06 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:17:43.524355 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e950dbf8-4710-4aea-9282-742957103d06 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:17:43.524484 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e950dbf8-4710-4aea-9282-742957103d06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:17:43.542600 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-e950dbf8-4710-4aea-9282-742957103d06 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:17:43.543797 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-e950dbf8-4710-4aea-9282-742957103d06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:17:43.552362 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 702/1396] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:17:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:17:43.596375 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756fcda2-963e-4188-8b36-ef40309b1c02 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:17:43.611446 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756fcda2-963e-4188-8b36-ef40309b1c02 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:17:43.617073 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76c2fef-b0bd-4aea-9b4b-1a99f95eb680 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '796aeeff16df4739bddec1f6dee61bfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:17:43.617710 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76c2fef-b0bd-4aea-9b4b-1a99f95eb680 service nova] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:17:43.618000 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76c2fef-b0bd-4aea-9b4b-1a99f95eb680 service nova] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:17:43.618266 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76c2fef-b0bd-4aea-9b4b-1a99f95eb680 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:17:43.626984 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76c2fef-b0bd-4aea-9b4b-1a99f95eb680 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:17:43.628640 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76c2fef-b0bd-4aea-9b4b-1a99f95eb680 service nova] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:17:43.638142 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 695/1397] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:17:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:17:43.882111 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6b02b4-c6d5-46b8-8aea-aaf6ce913fdb None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:17:43.882401 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6b02b4-c6d5-46b8-8aea-aaf6ce913fdb None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:17:43.882683 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6b02b4-c6d5-46b8-8aea-aaf6ce913fdb None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:17:43.884156 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 703/1398] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:17:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:17:43.890012 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b7772b-97bf-409b-a61e-b569efe1823d None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:17:43.890310 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b7772b-97bf-409b-a61e-b569efe1823d None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:17:43.891133 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b7772b-97bf-409b-a61e-b569efe1823d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:17:43.910245 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-55b7772b-97bf-409b-a61e-b569efe1823d None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:17:43.911788 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-55b7772b-97bf-409b-a61e-b569efe1823d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:17:43.920616 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 696/1399] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:17:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:17:43.977517 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d35a7ff-a45b-425a-8dd4-556143964e55 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:17:43.977828 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d35a7ff-a45b-425a-8dd4-556143964e55 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:17:43.978110 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d35a7ff-a45b-425a-8dd4-556143964e55 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:17:43.979469 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 704/1400] 10.222.0.44 () {54 vars in 962 bytes} [Wed Jun 29 11:17:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:17:43.984882 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a3404e-8f88-49ee-843b-2b3bc4be3c3f None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:17:43.985174 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a3404e-8f88-49ee-843b-2b3bc4be3c3f None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:17:43.985456 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a3404e-8f88-49ee-843b-2b3bc4be3c3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:17:44.035348 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-07a3404e-8f88-49ee-843b-2b3bc4be3c3f None None] MFA Rules not processed for user `ede3a17c7063432e8ee66462766584f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:17:44.036598 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-07a3404e-8f88-49ee-843b-2b3bc4be3c3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:17:44.068331 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:17:44.097184 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 697/1401] 10.222.0.44 () {58 vars in 1059 bytes} [Wed Jun 29 11:17:43 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:17:44.104893 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d35a7ff-a45b-425a-8dd4-556143964e55 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:17:44.116000 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d35a7ff-a45b-425a-8dd4-556143964e55 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:17:44.119544 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc33065d-6e59-406e-85ea-c6e11b4574b4 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede3a17c7063432e8ee66462766584f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:17:44.120402 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc33065d-6e59-406e-85ea-c6e11b4574b4 service glance] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:17:44.120690 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc33065d-6e59-406e-85ea-c6e11b4574b4 service glance] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:17:44.120973 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc33065d-6e59-406e-85ea-c6e11b4574b4 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:17:44.130054 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc33065d-6e59-406e-85ea-c6e11b4574b4 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:17:44.131487 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc33065d-6e59-406e-85ea-c6e11b4574b4 service glance] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:17:44.141482 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 705/1402] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:17:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:17:45.034596 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3029a7b-404a-4f5a-a734-788ab31806d1 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:17:45.035019 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3029a7b-404a-4f5a-a734-788ab31806d1 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:17:45.035157 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3029a7b-404a-4f5a-a734-788ab31806d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:17:45.083751 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-e3029a7b-404a-4f5a-a734-788ab31806d1 None None] MFA Rules not processed for user `55487c2107a44c15b656b4865ee0d4fa`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:17:45.084980 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-e3029a7b-404a-4f5a-a734-788ab31806d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:17:45.148146 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 698/1403] 10.222.0.44 () {58 vars in 1009 bytes} [Wed Jun 29 11:17:45 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:17:45.189821 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411fcdca-1784-4a5f-8415-d99e5d2af907 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:17:45.190277 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411fcdca-1784-4a5f-8415-d99e5d2af907 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:17:45.190400 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411fcdca-1784-4a5f-8415-d99e5d2af907 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:17:45.191749 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 706/1404] 10.222.0.44 () {54 vars in 942 bytes} [Wed Jun 29 11:17:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:17:45.198331 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe86d81-ad8c-474e-90c7-79277e37ed29 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:17:45.198694 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe86d81-ad8c-474e-90c7-79277e37ed29 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:17:45.199041 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe86d81-ad8c-474e-90c7-79277e37ed29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:17:45.246782 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-abe86d81-ad8c-474e-90c7-79277e37ed29 None None] MFA Rules not processed for user `e3b38493ba904759badc800e396befda`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:17:45.248028 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-abe86d81-ad8c-474e-90c7-79277e37ed29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:17:45.320570 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 699/1405] 10.222.0.44 () {58 vars in 1039 bytes} [Wed Jun 29 11:17:45 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:17:45.330874 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411fcdca-1784-4a5f-8415-d99e5d2af907 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:17:45.338454 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=61 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:17:45.345352 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411fcdca-1784-4a5f-8415-d99e5d2af907 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:17:45.350412 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bbf7f7-a054-4615-8703-217ac81352c0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3b38493ba904759badc800e396befda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:17:45.351195 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bbf7f7-a054-4615-8703-217ac81352c0 service swift] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:17:45.351480 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bbf7f7-a054-4615-8703-217ac81352c0 service swift] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:17:45.351764 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bbf7f7-a054-4615-8703-217ac81352c0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:17:45.360534 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bbf7f7-a054-4615-8703-217ac81352c0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:17:45.362110 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bbf7f7-a054-4615-8703-217ac81352c0 service swift] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:17:45.370261 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 707/1406] 10.222.0.44 () {60 vars in 1384 bytes} [Wed Jun 29 11:17:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 29 11:17:45.822992 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12572ff-a5da-4e96-b3ca-cc79e20b4586 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:17:45.823288 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12572ff-a5da-4e96-b3ca-cc79e20b4586 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:17:45.823573 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12572ff-a5da-4e96-b3ca-cc79e20b4586 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:17:45.824998 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 700/1407] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:17:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:17:45.830467 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8388f3da-38fb-4314-ba76-8758c68e713a None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:17:45.830763 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8388f3da-38fb-4314-ba76-8758c68e713a None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:17:45.831044 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8388f3da-38fb-4314-ba76-8758c68e713a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:17:45.854373 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-8388f3da-38fb-4314-ba76-8758c68e713a None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:17:45.855634 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-8388f3da-38fb-4314-ba76-8758c68e713a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:17:45.865163 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 708/1408] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:17:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:17:45.917597 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12572ff-a5da-4e96-b3ca-cc79e20b4586 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:17:45.941162 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12572ff-a5da-4e96-b3ca-cc79e20b4586 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:17:45.947294 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6842bdb3-52d9-4475-b9d4-6b4e88cda8de service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '796aeeff16df4739bddec1f6dee61bfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:17:45.948068 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6842bdb3-52d9-4475-b9d4-6b4e88cda8de service nova] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:17:45.948416 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6842bdb3-52d9-4475-b9d4-6b4e88cda8de service nova] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:17:45.948812 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6842bdb3-52d9-4475-b9d4-6b4e88cda8de service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:17:45.958582 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6842bdb3-52d9-4475-b9d4-6b4e88cda8de service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:17:45.960306 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6842bdb3-52d9-4475-b9d4-6b4e88cda8de service nova] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:17:45.965620 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:17:45.973813 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 701/1409] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:17:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:17:55.848748 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=8 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:18:09.081279 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5d2b6c-bc3d-440d-9480-2064afb011ad None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:18:09.081908 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5d2b6c-bc3d-440d-9480-2064afb011ad None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:18:09.081908 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5d2b6c-bc3d-440d-9480-2064afb011ad None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:18:09.083085 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 709/1410] 10.222.0.44 () {54 vars in 918 bytes} [Wed Jun 29 11:18:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:18:09.088448 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faf6ac0-c55b-483e-957e-9b955e88bdc5 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:18:09.088845 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faf6ac0-c55b-483e-957e-9b955e88bdc5 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:18:09.089006 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faf6ac0-c55b-483e-957e-9b955e88bdc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:18:09.106677 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-7faf6ac0-c55b-483e-957e-9b955e88bdc5 None None] MFA Rules not processed for user `1a0505342b704d02977a5125957c2c9e`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:18:09.107907 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-7faf6ac0-c55b-483e-957e-9b955e88bdc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:18:09.116290 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 702/1411] 10.222.0.44 () {58 vars in 1015 bytes} [Wed Jun 29 11:18:09 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:18:09.130326 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d150adab-9b13-4cf1-888c-3a41d3318317 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:18:09.130615 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d150adab-9b13-4cf1-888c-3a41d3318317 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:18:09.130897 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d150adab-9b13-4cf1-888c-3a41d3318317 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:18:09.132266 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 710/1412] 10.222.0.44 () {54 vars in 918 bytes} [Wed Jun 29 11:18:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:18:09.136830 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3991c13-8e27-4634-99f3-cd7ab75b94cf None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:18:09.137116 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3991c13-8e27-4634-99f3-cd7ab75b94cf None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:18:09.137393 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3991c13-8e27-4634-99f3-cd7ab75b94cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:18:09.155026 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-d3991c13-8e27-4634-99f3-cd7ab75b94cf None None] MFA Rules not processed for user `796aeeff16df4739bddec1f6dee61bfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:18:09.156234 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-d3991c13-8e27-4634-99f3-cd7ab75b94cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:18:09.165091 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 703/1413] 10.222.0.44 () {58 vars in 1015 bytes} [Wed Jun 29 11:18:09 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:18:11.649638 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105daefd-31de-4565-bdc3-bf7cf09e9dee None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:18:11.650174 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105daefd-31de-4565-bdc3-bf7cf09e9dee None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:18:11.650300 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105daefd-31de-4565-bdc3-bf7cf09e9dee None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:18:11.651646 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 711/1414] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 29 11:18:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:18:11.657102 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbb6a69-89b4-4b64-b1c9-74b1b7ec6b0e None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:18:11.657523 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbb6a69-89b4-4b64-b1c9-74b1b7ec6b0e None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:18:11.657659 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbb6a69-89b4-4b64-b1c9-74b1b7ec6b0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:18:11.677597 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-6cbb6a69-89b4-4b64-b1c9-74b1b7ec6b0e None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:18:11.679492 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-6cbb6a69-89b4-4b64-b1c9-74b1b7ec6b0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:18:11.686898 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 704/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 29 11:18:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:18:11.730731 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105daefd-31de-4565-bdc3-bf7cf09e9dee None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:18:11.745414 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105daefd-31de-4565-bdc3-bf7cf09e9dee None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:18:11.751501 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bdbc9f-9f1d-4e3d-b5af-8e6f43bc3b14 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '796aeeff16df4739bddec1f6dee61bfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:18:11.752164 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bdbc9f-9f1d-4e3d-b5af-8e6f43bc3b14 service nova] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:18:11.752449 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bdbc9f-9f1d-4e3d-b5af-8e6f43bc3b14 service nova] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:18:11.752727 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bdbc9f-9f1d-4e3d-b5af-8e6f43bc3b14 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:18:11.761077 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bdbc9f-9f1d-4e3d-b5af-8e6f43bc3b14 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:18:11.762683 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bdbc9f-9f1d-4e3d-b5af-8e6f43bc3b14 service nova] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:18:11.772071 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 712/1416] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:18:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:18:12.673385 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efa0b08-23e9-49bf-880d-bac8cd09c57b None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:18:12.673934 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efa0b08-23e9-49bf-880d-bac8cd09c57b None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:18:12.673934 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efa0b08-23e9-49bf-880d-bac8cd09c57b None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:18:12.675248 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 705/1417] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 29 11:18:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:18:12.680175 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d826d7-ed10-4060-ad44-2f1dd2083372 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:18:12.680554 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d826d7-ed10-4060-ad44-2f1dd2083372 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:18:12.680742 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d826d7-ed10-4060-ad44-2f1dd2083372 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:18:12.698417 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-67d826d7-ed10-4060-ad44-2f1dd2083372 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:18:12.699648 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-67d826d7-ed10-4060-ad44-2f1dd2083372 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:18:12.707876 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 713/1418] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 29 11:18:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:18:12.858304 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efa0b08-23e9-49bf-880d-bac8cd09c57b None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:18:12.872377 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efa0b08-23e9-49bf-880d-bac8cd09c57b None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:18:12.878354 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d260babd-4091-4202-bb91-91e507ac47b9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '796aeeff16df4739bddec1f6dee61bfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:18:12.878986 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d260babd-4091-4202-bb91-91e507ac47b9 service nova] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:18:12.879270 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d260babd-4091-4202-bb91-91e507ac47b9 service nova] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:18:12.879545 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d260babd-4091-4202-bb91-91e507ac47b9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:18:12.887570 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d260babd-4091-4202-bb91-91e507ac47b9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:18:12.889177 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d260babd-4091-4202-bb91-91e507ac47b9 service nova] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:18:12.898949 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 706/1419] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:18:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:18:13.781223 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0e0265-c879-4608-a562-c1830a1da551 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:18:13.781775 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0e0265-c879-4608-a562-c1830a1da551 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:18:13.781775 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0e0265-c879-4608-a562-c1830a1da551 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:18:13.783140 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 714/1420] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 29 11:18:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:18:13.787928 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afea5029-73ae-4a9c-8869-624451d0c0d1 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:18:13.788304 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afea5029-73ae-4a9c-8869-624451d0c0d1 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:18:13.788497 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afea5029-73ae-4a9c-8869-624451d0c0d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:18:13.805951 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-afea5029-73ae-4a9c-8869-624451d0c0d1 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:18:13.807137 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-afea5029-73ae-4a9c-8869-624451d0c0d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:18:13.815459 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 707/1421] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 29 11:18:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:18:13.967240 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0e0265-c879-4608-a562-c1830a1da551 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:18:13.986808 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0e0265-c879-4608-a562-c1830a1da551 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:18:13.994706 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3872b456-a4bd-47ad-bb31-25bc1f22f45f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '796aeeff16df4739bddec1f6dee61bfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:18:13.995351 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3872b456-a4bd-47ad-bb31-25bc1f22f45f service nova] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:18:13.995637 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3872b456-a4bd-47ad-bb31-25bc1f22f45f service nova] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:18:13.995951 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3872b456-a4bd-47ad-bb31-25bc1f22f45f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:18:14.004889 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3872b456-a4bd-47ad-bb31-25bc1f22f45f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:18:14.006482 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3872b456-a4bd-47ad-bb31-25bc1f22f45f service nova] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:18:14.015850 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 715/1422] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:18:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:18:14.873551 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac364e7-7e28-4708-9921-8ea1f389e03d None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:18:14.874135 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac364e7-7e28-4708-9921-8ea1f389e03d None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:18:14.874135 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac364e7-7e28-4708-9921-8ea1f389e03d None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:18:14.876099 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 708/1423] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 29 11:18:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:18:14.881226 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb0124c-dc62-4969-97b3-0ffa5d5483e4 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:18:14.881669 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb0124c-dc62-4969-97b3-0ffa5d5483e4 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:18:14.881845 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb0124c-dc62-4969-97b3-0ffa5d5483e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:18:14.901630 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-cbb0124c-dc62-4969-97b3-0ffa5d5483e4 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:18:14.902838 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-cbb0124c-dc62-4969-97b3-0ffa5d5483e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:18:14.914306 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 716/1424] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 29 11:18:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:18:15.078309 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac364e7-7e28-4708-9921-8ea1f389e03d None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:18:15.092768 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac364e7-7e28-4708-9921-8ea1f389e03d None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:18:15.099645 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30df6b96-8a6c-4899-8185-5248f0d571c8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '796aeeff16df4739bddec1f6dee61bfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:18:15.100338 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30df6b96-8a6c-4899-8185-5248f0d571c8 service nova] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:18:15.100622 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30df6b96-8a6c-4899-8185-5248f0d571c8 service nova] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:18:15.100894 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30df6b96-8a6c-4899-8185-5248f0d571c8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:18:15.109324 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30df6b96-8a6c-4899-8185-5248f0d571c8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:18:15.110879 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30df6b96-8a6c-4899-8185-5248f0d571c8 service nova] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:18:15.121865 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 709/1425] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:18:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:18:16.005016 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2c4331-2f83-47fa-8a80-642599867234 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:18:16.005562 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2c4331-2f83-47fa-8a80-642599867234 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:18:16.005562 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2c4331-2f83-47fa-8a80-642599867234 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:18:16.006925 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 717/1426] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 29 11:18:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:18:16.011893 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186ceb39-7c3c-4089-9da4-560e00bbb15f None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:18:16.012268 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186ceb39-7c3c-4089-9da4-560e00bbb15f None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:18:16.012454 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186ceb39-7c3c-4089-9da4-560e00bbb15f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:18:16.030552 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-186ceb39-7c3c-4089-9da4-560e00bbb15f None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:18:16.031751 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-186ceb39-7c3c-4089-9da4-560e00bbb15f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:18:16.039497 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 710/1427] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 29 11:18:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:18:16.190392 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2c4331-2f83-47fa-8a80-642599867234 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:18:16.204766 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2c4331-2f83-47fa-8a80-642599867234 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:18:16.210061 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c304c93-7341-4176-b6a8-d830b09352f9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '796aeeff16df4739bddec1f6dee61bfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:18:16.210725 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c304c93-7341-4176-b6a8-d830b09352f9 service nova] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:18:16.211008 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c304c93-7341-4176-b6a8-d830b09352f9 service nova] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:18:16.211284 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c304c93-7341-4176-b6a8-d830b09352f9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:18:16.219760 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c304c93-7341-4176-b6a8-d830b09352f9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:18:16.221415 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c304c93-7341-4176-b6a8-d830b09352f9 service nova] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:18:16.230579 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 718/1428] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:18:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:18:17.111644 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fd503d-0b5a-46a9-9091-80257293848b None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:18:17.112231 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fd503d-0b5a-46a9-9091-80257293848b None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:18:17.112231 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fd503d-0b5a-46a9-9091-80257293848b None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:18:17.113614 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 711/1429] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 29 11:18:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:18:17.118522 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09b9fd5-bc3c-4bb7-90ae-f61ed862cdf1 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:18:17.118941 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09b9fd5-bc3c-4bb7-90ae-f61ed862cdf1 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:18:17.119078 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09b9fd5-bc3c-4bb7-90ae-f61ed862cdf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:18:17.137957 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-d09b9fd5-bc3c-4bb7-90ae-f61ed862cdf1 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:18:17.139153 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-d09b9fd5-bc3c-4bb7-90ae-f61ed862cdf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:18:17.146796 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 719/1430] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 29 11:18:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:18:17.295197 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fd503d-0b5a-46a9-9091-80257293848b None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:18:17.310256 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fd503d-0b5a-46a9-9091-80257293848b None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:18:17.315599 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9584c09-fe08-45bb-bdf3-43b457119372 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '796aeeff16df4739bddec1f6dee61bfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:18:17.316261 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9584c09-fe08-45bb-bdf3-43b457119372 service nova] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:18:17.316549 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9584c09-fe08-45bb-bdf3-43b457119372 service nova] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:18:17.316827 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9584c09-fe08-45bb-bdf3-43b457119372 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:18:17.324835 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9584c09-fe08-45bb-bdf3-43b457119372 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:18:17.326764 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9584c09-fe08-45bb-bdf3-43b457119372 service nova] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:18:17.336430 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 712/1431] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:18:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:18:18.345163 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9f899c-4ee7-4cad-a9f1-707929b8a883 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:18:18.345756 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9f899c-4ee7-4cad-a9f1-707929b8a883 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:18:18.345756 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9f899c-4ee7-4cad-a9f1-707929b8a883 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:18:18.347097 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 720/1432] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 29 11:18:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:18:18.352045 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17d99d8-e980-4e8d-9824-e93df425965a None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:18:18.352445 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17d99d8-e980-4e8d-9824-e93df425965a None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:18:18.352649 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17d99d8-e980-4e8d-9824-e93df425965a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:18:18.370418 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-d17d99d8-e980-4e8d-9824-e93df425965a None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:18:18.371601 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-d17d99d8-e980-4e8d-9824-e93df425965a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:18:18.379936 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 713/1433] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 29 11:18:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:18:18.528589 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9f899c-4ee7-4cad-a9f1-707929b8a883 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:18:18.542625 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9f899c-4ee7-4cad-a9f1-707929b8a883 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:18:18.548335 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384c3a2a-e152-401b-9367-d4cb7354c5ed service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '796aeeff16df4739bddec1f6dee61bfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:18:18.548967 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384c3a2a-e152-401b-9367-d4cb7354c5ed service nova] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:18:18.549249 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384c3a2a-e152-401b-9367-d4cb7354c5ed service nova] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:18:18.549518 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384c3a2a-e152-401b-9367-d4cb7354c5ed service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:18:18.557377 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384c3a2a-e152-401b-9367-d4cb7354c5ed service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:18:18.558947 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384c3a2a-e152-401b-9367-d4cb7354c5ed service nova] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:18:18.568742 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 721/1434] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:18:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:18:19.442082 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58958b31-a179-4375-a37d-2b51209cc131 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:18:19.442664 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58958b31-a179-4375-a37d-2b51209cc131 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:18:19.442664 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58958b31-a179-4375-a37d-2b51209cc131 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:18:19.444105 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 714/1435] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 29 11:18:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:18:19.448930 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639c6f1f-a0ac-4b74-95f1-e916a1fc3c3a None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:18:19.449371 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639c6f1f-a0ac-4b74-95f1-e916a1fc3c3a None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:18:19.449491 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639c6f1f-a0ac-4b74-95f1-e916a1fc3c3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:18:19.467762 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-639c6f1f-a0ac-4b74-95f1-e916a1fc3c3a None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:18:19.469589 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-639c6f1f-a0ac-4b74-95f1-e916a1fc3c3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:18:19.477233 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 722/1436] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 29 11:18:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:18:19.627688 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58958b31-a179-4375-a37d-2b51209cc131 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:18:19.641747 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58958b31-a179-4375-a37d-2b51209cc131 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:18:19.647021 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4900f69b-4a61-498e-a31b-393229659731 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '796aeeff16df4739bddec1f6dee61bfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:18:19.647644 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4900f69b-4a61-498e-a31b-393229659731 service nova] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:18:19.647968 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4900f69b-4a61-498e-a31b-393229659731 service nova] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:18:19.648249 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4900f69b-4a61-498e-a31b-393229659731 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:18:19.656584 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4900f69b-4a61-498e-a31b-393229659731 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:18:19.658134 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4900f69b-4a61-498e-a31b-393229659731 service nova] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:18:19.667260 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 715/1437] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:18:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:18:20.545267 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dc3504-2049-4a26-9c5f-afdcdf3eb2d3 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:18:20.545804 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dc3504-2049-4a26-9c5f-afdcdf3eb2d3 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:18:20.545804 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dc3504-2049-4a26-9c5f-afdcdf3eb2d3 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:18:20.547124 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 723/1438] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 29 11:18:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:18:20.552124 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5763cddd-ee7e-469e-ab1e-6b9cde33be36 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:18:20.552543 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5763cddd-ee7e-469e-ab1e-6b9cde33be36 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:18:20.552709 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5763cddd-ee7e-469e-ab1e-6b9cde33be36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:18:20.571133 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-5763cddd-ee7e-469e-ab1e-6b9cde33be36 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:18:20.572362 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-5763cddd-ee7e-469e-ab1e-6b9cde33be36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:18:20.580561 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 716/1439] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 29 11:18:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 11:18:20.730122 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dc3504-2049-4a26-9c5f-afdcdf3eb2d3 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:18:20.744384 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dc3504-2049-4a26-9c5f-afdcdf3eb2d3 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:18:20.749619 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942461fc-75a9-426f-a51e-8425e191e21c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '796aeeff16df4739bddec1f6dee61bfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:18:20.750244 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942461fc-75a9-426f-a51e-8425e191e21c service nova] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:18:20.750524 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942461fc-75a9-426f-a51e-8425e191e21c service nova] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:18:20.750799 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942461fc-75a9-426f-a51e-8425e191e21c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:18:20.758652 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942461fc-75a9-426f-a51e-8425e191e21c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:18:20.760283 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942461fc-75a9-426f-a51e-8425e191e21c service nova] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:18:20.769796 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 724/1440] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:18:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:18:22.692472 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e065e83-0e29-4dcf-867a-bda39cb8f6fc None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:18:22.693010 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e065e83-0e29-4dcf-867a-bda39cb8f6fc None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:18:22.693010 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e065e83-0e29-4dcf-867a-bda39cb8f6fc None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:18:22.694406 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 717/1441] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 29 11:18:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:18:22.699635 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a957f255-5c8b-42fe-9de5-b7d58618d35d None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:18:22.700142 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a957f255-5c8b-42fe-9de5-b7d58618d35d None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:18:22.700229 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a957f255-5c8b-42fe-9de5-b7d58618d35d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:18:22.718701 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-a957f255-5c8b-42fe-9de5-b7d58618d35d None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:18:22.719942 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-a957f255-5c8b-42fe-9de5-b7d58618d35d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:18:22.727647 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 725/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 29 11:18:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:18:22.766452 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e065e83-0e29-4dcf-867a-bda39cb8f6fc None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:18:22.777037 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e065e83-0e29-4dcf-867a-bda39cb8f6fc None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:18:22.780468 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01ab3df-feb6-497e-a693-e626c41033b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f4bb9f989e4024a5da28dced646b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:18:22.781096 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01ab3df-feb6-497e-a693-e626c41033b3 service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:18:22.781381 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01ab3df-feb6-497e-a693-e626c41033b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:18:22.781659 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01ab3df-feb6-497e-a693-e626c41033b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:18:22.790322 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01ab3df-feb6-497e-a693-e626c41033b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:18:22.791725 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01ab3df-feb6-497e-a693-e626c41033b3 service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:18:22.800937 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 718/1443] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:18:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 11:18:24.981754 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d247619-3879-4b02-98ef-c9ca035686f5 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:18:24.982263 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d247619-3879-4b02-98ef-c9ca035686f5 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:18:24.982368 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d247619-3879-4b02-98ef-c9ca035686f5 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:18:24.983740 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 726/1444] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 29 11:18:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:18:24.989184 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c469dee5-5ba2-4e62-9a5e-5db1d722f79b None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:18:24.989613 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c469dee5-5ba2-4e62-9a5e-5db1d722f79b None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:18:24.989720 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c469dee5-5ba2-4e62-9a5e-5db1d722f79b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:18:25.007503 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-c469dee5-5ba2-4e62-9a5e-5db1d722f79b None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:18:25.008732 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-c469dee5-5ba2-4e62-9a5e-5db1d722f79b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:18:25.016994 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 719/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 29 11:18:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 11:18:25.054481 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d247619-3879-4b02-98ef-c9ca035686f5 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:18:25.065448 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d247619-3879-4b02-98ef-c9ca035686f5 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:18:25.068872 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505371a0-dae6-475d-a4f5-e7e2aefc3273 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f4bb9f989e4024a5da28dced646b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:18:25.069499 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505371a0-dae6-475d-a4f5-e7e2aefc3273 service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:18:25.069778 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505371a0-dae6-475d-a4f5-e7e2aefc3273 service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:18:25.070053 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505371a0-dae6-475d-a4f5-e7e2aefc3273 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:18:25.079510 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505371a0-dae6-475d-a4f5-e7e2aefc3273 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:18:25.081024 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505371a0-dae6-475d-a4f5-e7e2aefc3273 service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:18:25.090830 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 727/1446] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:18:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:18:27.482022 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bc8abc-58ad-416a-a725-9288ffeec3fe None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:18:27.482599 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bc8abc-58ad-416a-a725-9288ffeec3fe None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:18:27.482599 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bc8abc-58ad-416a-a725-9288ffeec3fe None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:18:27.484523 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 720/1447] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 29 11:18:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:18:27.489689 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6a695a-2d6d-4e11-8486-577398fac570 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:18:27.490099 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6a695a-2d6d-4e11-8486-577398fac570 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:18:27.490249 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6a695a-2d6d-4e11-8486-577398fac570 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:18:27.508268 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-2d6a695a-2d6d-4e11-8486-577398fac570 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:18:27.509462 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-2d6a695a-2d6d-4e11-8486-577398fac570 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:18:27.517515 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 728/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 29 11:18:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:18:27.555682 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bc8abc-58ad-416a-a725-9288ffeec3fe None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:18:27.566749 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bc8abc-58ad-416a-a725-9288ffeec3fe None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:18:27.570250 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9986a5c5-380b-4889-9ba8-f868baf61c89 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f4bb9f989e4024a5da28dced646b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:18:27.570881 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9986a5c5-380b-4889-9ba8-f868baf61c89 service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:18:27.571166 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9986a5c5-380b-4889-9ba8-f868baf61c89 service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:18:27.571443 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9986a5c5-380b-4889-9ba8-f868baf61c89 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:18:27.580772 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9986a5c5-380b-4889-9ba8-f868baf61c89 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:18:27.582177 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9986a5c5-380b-4889-9ba8-f868baf61c89 service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:18:27.591635 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 721/1449] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:18:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:18:29.726151 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a307c02-b351-4447-9304-6e826c99ba9d None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:18:29.726739 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a307c02-b351-4447-9304-6e826c99ba9d None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:18:29.726739 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a307c02-b351-4447-9304-6e826c99ba9d None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:18:29.728110 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 729/1450] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 29 11:18:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:18:29.734122 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf24c727-9f7e-4098-8a2e-3c9e5f4c16c1 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:18:29.734576 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf24c727-9f7e-4098-8a2e-3c9e5f4c16c1 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:18:29.734687 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf24c727-9f7e-4098-8a2e-3c9e5f4c16c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:18:29.759507 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-bf24c727-9f7e-4098-8a2e-3c9e5f4c16c1 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:18:29.760734 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-bf24c727-9f7e-4098-8a2e-3c9e5f4c16c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:18:29.770723 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 722/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 29 11:18:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:18:29.809561 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a307c02-b351-4447-9304-6e826c99ba9d None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:18:29.821332 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a307c02-b351-4447-9304-6e826c99ba9d None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:18:29.825097 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68359d4b-c309-43f1-af87-aed8f5f09b72 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f4bb9f989e4024a5da28dced646b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:18:29.825928 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68359d4b-c309-43f1-af87-aed8f5f09b72 service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:18:29.826417 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68359d4b-c309-43f1-af87-aed8f5f09b72 service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:18:29.826899 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68359d4b-c309-43f1-af87-aed8f5f09b72 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:18:29.836758 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68359d4b-c309-43f1-af87-aed8f5f09b72 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:18:29.838676 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68359d4b-c309-43f1-af87-aed8f5f09b72 service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:18:29.849089 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 730/1452] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:18:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:18:31.926447 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa4a9d8-f483-44ce-93d5-09b2467c0095 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:18:31.926907 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa4a9d8-f483-44ce-93d5-09b2467c0095 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:18:31.927019 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa4a9d8-f483-44ce-93d5-09b2467c0095 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:18:31.928419 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 723/1453] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 29 11:18:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:18:31.933673 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134f8a73-eaea-4629-955a-cbdb3289bbba None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:18:31.934114 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134f8a73-eaea-4629-955a-cbdb3289bbba None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:18:31.934243 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134f8a73-eaea-4629-955a-cbdb3289bbba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:18:31.951586 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-134f8a73-eaea-4629-955a-cbdb3289bbba None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:18:31.953336 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-134f8a73-eaea-4629-955a-cbdb3289bbba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:18:31.961022 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 731/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 29 11:18:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:18:31.999125 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa4a9d8-f483-44ce-93d5-09b2467c0095 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:18:32.000862 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-3aa4a9d8-f483-44ce-93d5-09b2467c0095 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:18:32.013651 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa4a9d8-f483-44ce-93d5-09b2467c0095 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:18:32.017663 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72f7a5f-7c99-456d-b74c-8e8da6c4be78 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f4bb9f989e4024a5da28dced646b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:18:32.018293 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72f7a5f-7c99-456d-b74c-8e8da6c4be78 service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:18:32.018576 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72f7a5f-7c99-456d-b74c-8e8da6c4be78 service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:18:32.018855 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72f7a5f-7c99-456d-b74c-8e8da6c4be78 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:18:32.027562 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72f7a5f-7c99-456d-b74c-8e8da6c4be78 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:18:32.029007 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72f7a5f-7c99-456d-b74c-8e8da6c4be78 service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:18:32.040200 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 724/1455] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:18:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:18:34.214405 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de7c0a2-c90e-441f-804f-c9722e93959e None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:18:34.214807 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de7c0a2-c90e-441f-804f-c9722e93959e None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:18:34.214956 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de7c0a2-c90e-441f-804f-c9722e93959e None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:18:34.216360 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 732/1456] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 29 11:18:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:18:34.221591 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4e3aae-436f-48a5-b605-d73d8fa515dd None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:18:34.221981 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4e3aae-436f-48a5-b605-d73d8fa515dd None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:18:34.222155 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4e3aae-436f-48a5-b605-d73d8fa515dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:18:34.240283 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-2f4e3aae-436f-48a5-b605-d73d8fa515dd None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:18:34.241471 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-2f4e3aae-436f-48a5-b605-d73d8fa515dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:18:34.249047 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 725/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 29 11:18:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:18:34.285623 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de7c0a2-c90e-441f-804f-c9722e93959e None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:18:34.296588 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de7c0a2-c90e-441f-804f-c9722e93959e None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:18:34.300020 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd3ab47-5307-4879-b3dc-3a46fc4e03c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f4bb9f989e4024a5da28dced646b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:18:34.300647 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd3ab47-5307-4879-b3dc-3a46fc4e03c4 service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:18:34.300927 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd3ab47-5307-4879-b3dc-3a46fc4e03c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:18:34.301202 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd3ab47-5307-4879-b3dc-3a46fc4e03c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:18:34.309808 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd3ab47-5307-4879-b3dc-3a46fc4e03c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:18:34.311220 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd3ab47-5307-4879-b3dc-3a46fc4e03c4 service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:18:34.321133 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 733/1458] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:18:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:18:36.392487 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d056de-cc76-42fb-aea4-98e52d022c41 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:18:36.393022 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d056de-cc76-42fb-aea4-98e52d022c41 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:18:36.393022 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d056de-cc76-42fb-aea4-98e52d022c41 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:18:36.394673 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 726/1459] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 29 11:18:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:18:36.399661 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cb946f-7dad-4250-ad61-270a6a52b59f None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:18:36.400128 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cb946f-7dad-4250-ad61-270a6a52b59f None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:18:36.400261 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cb946f-7dad-4250-ad61-270a6a52b59f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:18:36.406251 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=70 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:18:36.417986 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-52cb946f-7dad-4250-ad61-270a6a52b59f None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:18:36.419181 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-52cb946f-7dad-4250-ad61-270a6a52b59f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:18:36.427491 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 734/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 29 11:18:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:18:36.469440 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d056de-cc76-42fb-aea4-98e52d022c41 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:18:36.476814 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d056de-cc76-42fb-aea4-98e52d022c41 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:18:36.480504 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ff7467-d9c3-474a-8bcb-801f55f35949 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f4bb9f989e4024a5da28dced646b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:18:36.481219 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ff7467-d9c3-474a-8bcb-801f55f35949 service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:18:36.481503 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ff7467-d9c3-474a-8bcb-801f55f35949 service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:18:36.481776 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ff7467-d9c3-474a-8bcb-801f55f35949 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:18:36.489186 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=75 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:18:36.491468 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ff7467-d9c3-474a-8bcb-801f55f35949 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:18:36.492983 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ff7467-d9c3-474a-8bcb-801f55f35949 service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:18:36.502818 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 727/1461] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:18:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:18:38.670681 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f09f7a-f6d3-4256-8534-387dc473fb9e None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:18:38.671193 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f09f7a-f6d3-4256-8534-387dc473fb9e None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:18:38.671483 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f09f7a-f6d3-4256-8534-387dc473fb9e None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:18:38.672898 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 735/1462] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 29 11:18:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:18:38.678002 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f615939-828a-4b81-aaee-75a850496263 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:18:38.678452 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f615939-828a-4b81-aaee-75a850496263 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:18:38.678558 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f615939-828a-4b81-aaee-75a850496263 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:18:38.696047 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-9f615939-828a-4b81-aaee-75a850496263 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:18:38.697284 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-9f615939-828a-4b81-aaee-75a850496263 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:18:38.705549 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 728/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 29 11:18:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:18:38.743484 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f09f7a-f6d3-4256-8534-387dc473fb9e None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:18:38.753996 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f09f7a-f6d3-4256-8534-387dc473fb9e None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:18:38.757482 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeeefcfc-b640-42fe-a093-fd162d2517ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f4bb9f989e4024a5da28dced646b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:18:38.758110 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeefcfc-b640-42fe-a093-fd162d2517ff service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:18:38.758394 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeefcfc-b640-42fe-a093-fd162d2517ff service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:18:38.758673 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeefcfc-b640-42fe-a093-fd162d2517ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:18:38.768022 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeeefcfc-b640-42fe-a093-fd162d2517ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:18:38.769446 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeeefcfc-b640-42fe-a093-fd162d2517ff service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:18:38.778759 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 736/1464] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:18:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:18:40.805998 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9799f43-8dba-4dc7-ba3d-bff0397cead8 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:18:40.806566 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9799f43-8dba-4dc7-ba3d-bff0397cead8 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:18:40.806566 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9799f43-8dba-4dc7-ba3d-bff0397cead8 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:18:40.807944 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 729/1465] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 29 11:18:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:18:40.813525 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0414ed58-c18b-4c97-bdb4-37c196736f2b None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:18:40.813925 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0414ed58-c18b-4c97-bdb4-37c196736f2b None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:18:40.814080 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0414ed58-c18b-4c97-bdb4-37c196736f2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:18:40.831878 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-0414ed58-c18b-4c97-bdb4-37c196736f2b None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:18:40.833071 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-0414ed58-c18b-4c97-bdb4-37c196736f2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:18:40.840956 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 737/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 29 11:18:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:18:40.877647 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9799f43-8dba-4dc7-ba3d-bff0397cead8 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:18:40.888136 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9799f43-8dba-4dc7-ba3d-bff0397cead8 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:18:40.891570 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e47b1d-8e52-4199-beac-772e9a2ba858 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f4bb9f989e4024a5da28dced646b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:18:40.892230 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e47b1d-8e52-4199-beac-772e9a2ba858 service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:18:40.892513 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e47b1d-8e52-4199-beac-772e9a2ba858 service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:18:40.892788 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e47b1d-8e52-4199-beac-772e9a2ba858 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:18:40.901401 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e47b1d-8e52-4199-beac-772e9a2ba858 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:18:40.902827 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e47b1d-8e52-4199-beac-772e9a2ba858 service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:18:40.912068 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 730/1467] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:18:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:18:42.997353 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cc494a-aab2-4031-8587-84b5b6f5af16 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:18:42.997849 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cc494a-aab2-4031-8587-84b5b6f5af16 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:18:42.997960 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cc494a-aab2-4031-8587-84b5b6f5af16 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:18:42.999781 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 738/1468] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 29 11:18:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:18:43.005106 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8474ec-6eff-43ba-a707-e1b5f16fdfe4 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:18:43.005549 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8474ec-6eff-43ba-a707-e1b5f16fdfe4 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:18:43.005657 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8474ec-6eff-43ba-a707-e1b5f16fdfe4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:18:43.024371 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-ab8474ec-6eff-43ba-a707-e1b5f16fdfe4 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:18:43.031239 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-ab8474ec-6eff-43ba-a707-e1b5f16fdfe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:18:43.033445 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 731/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 29 11:18:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:18:43.069266 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cc494a-aab2-4031-8587-84b5b6f5af16 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:18:43.080406 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cc494a-aab2-4031-8587-84b5b6f5af16 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:18:43.084125 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9590050-9ad2-4532-8ad1-3d4c4343c826 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f4bb9f989e4024a5da28dced646b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:18:43.084829 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9590050-9ad2-4532-8ad1-3d4c4343c826 service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:18:43.085115 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9590050-9ad2-4532-8ad1-3d4c4343c826 service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:18:43.085391 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9590050-9ad2-4532-8ad1-3d4c4343c826 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:18:43.095441 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9590050-9ad2-4532-8ad1-3d4c4343c826 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:18:43.096955 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9590050-9ad2-4532-8ad1-3d4c4343c826 service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:18:43.110343 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 739/1470] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:18:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:18:53.018645 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=15 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:18:53.103439 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=17 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:19:02.985073 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0c5b3e-edb6-4aa0-9bf7-57d15d60b9bc None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:19:02.985680 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0c5b3e-edb6-4aa0-9bf7-57d15d60b9bc None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:19:02.985680 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0c5b3e-edb6-4aa0-9bf7-57d15d60b9bc None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:19:02.986976 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 732/1471] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:19:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:19:02.992423 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de466080-cc5a-4119-ad4f-33dc29dd61e0 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:19:02.992803 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de466080-cc5a-4119-ad4f-33dc29dd61e0 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:19:02.992979 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de466080-cc5a-4119-ad4f-33dc29dd61e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:19:03.011273 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-de466080-cc5a-4119-ad4f-33dc29dd61e0 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:19:03.012633 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-de466080-cc5a-4119-ad4f-33dc29dd61e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:19:03.020304 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 740/1472] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:19:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:19:03.071005 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3b2ce9-6b57-449a-8e7c-99bb9ef96260 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:19:03.072942 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3b2ce9-6b57-449a-8e7c-99bb9ef96260 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:19:03.072942 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3b2ce9-6b57-449a-8e7c-99bb9ef96260 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:19:03.073146 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 733/1473] 10.222.0.44 () {54 vars in 962 bytes} [Wed Jun 29 11:19:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:19:03.077747 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ef3240-1e09-4868-ad18-ef77f3d70a6c None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:19:03.078039 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ef3240-1e09-4868-ad18-ef77f3d70a6c None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:19:03.078321 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ef3240-1e09-4868-ad18-ef77f3d70a6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:19:03.097156 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-34ef3240-1e09-4868-ad18-ef77f3d70a6c None None] MFA Rules not processed for user `ede3a17c7063432e8ee66462766584f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:19:03.098355 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-34ef3240-1e09-4868-ad18-ef77f3d70a6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:19:03.104751 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 741/1474] 10.222.0.44 () {58 vars in 1059 bytes} [Wed Jun 29 11:19:03 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:19:03.112499 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3b2ce9-6b57-449a-8e7c-99bb9ef96260 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:19:03.122866 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3b2ce9-6b57-449a-8e7c-99bb9ef96260 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:19:03.126404 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da18257-bd3a-49d0-ae24-b518e383a59f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede3a17c7063432e8ee66462766584f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:19:03.127016 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da18257-bd3a-49d0-ae24-b518e383a59f service glance] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:19:03.127297 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da18257-bd3a-49d0-ae24-b518e383a59f service glance] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:19:03.127573 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da18257-bd3a-49d0-ae24-b518e383a59f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:19:03.136320 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da18257-bd3a-49d0-ae24-b518e383a59f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:19:03.137720 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da18257-bd3a-49d0-ae24-b518e383a59f service glance] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:19:03.146900 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 734/1475] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:19:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:19:05.260080 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e2b040-9e0b-4dd7-8287-6b289090396e None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:19:05.260634 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e2b040-9e0b-4dd7-8287-6b289090396e None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:19:05.260634 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e2b040-9e0b-4dd7-8287-6b289090396e None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:19:05.261980 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 742/1476] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:19:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:19:05.267900 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7a1f8f-981d-43be-bc7c-9f85d15428d3 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:19:05.268318 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7a1f8f-981d-43be-bc7c-9f85d15428d3 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:19:05.268463 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7a1f8f-981d-43be-bc7c-9f85d15428d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:19:05.285610 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-1d7a1f8f-981d-43be-bc7c-9f85d15428d3 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:19:05.286801 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-1d7a1f8f-981d-43be-bc7c-9f85d15428d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:19:05.294496 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 735/1477] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:19:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:19:05.317803 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e2b040-9e0b-4dd7-8287-6b289090396e None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:19:05.330485 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e2b040-9e0b-4dd7-8287-6b289090396e None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:19:05.334801 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58db29e8-26ac-4ed8-b89a-bda37e17d966 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:19:05.335649 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58db29e8-26ac-4ed8-b89a-bda37e17d966 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:19:05.335965 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58db29e8-26ac-4ed8-b89a-bda37e17d966 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:19:05.336241 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58db29e8-26ac-4ed8-b89a-bda37e17d966 admin admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:19:05.336972 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58db29e8-26ac-4ed8-b89a-bda37e17d966 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:19:05.338684 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: WARNING py.warnings [None req-58db29e8-26ac-4ed8-b89a-bda37e17d966 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 29 11:19:05.338684 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: warnings.warn(msg) Jun 29 11:19:05.338684 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: Jun 29 11:19:05.340094 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58db29e8-26ac-4ed8-b89a-bda37e17d966 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:19:05.358575 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 743/1478] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jun 29 11:19:05 2022] GET /identity/v3/projects => generated 2553 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 11:19:07.389898 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c36b38-6539-46b2-aa78-a1b1ae0624aa None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:19:07.390453 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c36b38-6539-46b2-aa78-a1b1ae0624aa None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:19:07.390453 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c36b38-6539-46b2-aa78-a1b1ae0624aa None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:19:07.391858 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 736/1479] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:19:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:19:07.397193 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a5e579-a1e1-4a17-b924-e80232f178e3 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:19:07.397635 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a5e579-a1e1-4a17-b924-e80232f178e3 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:19:07.397733 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a5e579-a1e1-4a17-b924-e80232f178e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:19:07.415119 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-27a5e579-a1e1-4a17-b924-e80232f178e3 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:19:07.416352 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-27a5e579-a1e1-4a17-b924-e80232f178e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:19:07.424550 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 744/1480] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:19:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:19:07.467896 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c36b38-6539-46b2-aa78-a1b1ae0624aa None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:19:07.481894 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c36b38-6539-46b2-aa78-a1b1ae0624aa None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:19:07.487747 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f334e15d-3828-4c14-8a92-e2fcdf207dbf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '796aeeff16df4739bddec1f6dee61bfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:19:07.488404 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f334e15d-3828-4c14-8a92-e2fcdf207dbf service nova] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:19:07.488687 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f334e15d-3828-4c14-8a92-e2fcdf207dbf service nova] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:19:07.488963 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f334e15d-3828-4c14-8a92-e2fcdf207dbf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:19:07.496935 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f334e15d-3828-4c14-8a92-e2fcdf207dbf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:19:07.498515 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f334e15d-3828-4c14-8a92-e2fcdf207dbf service nova] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:19:07.508565 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 737/1481] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:19:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:19:09.813261 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a629243f-d13f-4cd0-8a86-5b083b3e7f7a None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:19:09.813821 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a629243f-d13f-4cd0-8a86-5b083b3e7f7a None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:19:09.813821 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a629243f-d13f-4cd0-8a86-5b083b3e7f7a None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:19:09.815162 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 745/1482] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:19:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:19:09.820358 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99e2c76-fd22-4e39-b79d-15987b2e68f0 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:19:09.820805 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99e2c76-fd22-4e39-b79d-15987b2e68f0 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:19:09.820919 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99e2c76-fd22-4e39-b79d-15987b2e68f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:19:09.838463 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-a99e2c76-fd22-4e39-b79d-15987b2e68f0 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:19:09.839942 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-a99e2c76-fd22-4e39-b79d-15987b2e68f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:19:09.847589 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 738/1483] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:19:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:19:09.882522 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a629243f-d13f-4cd0-8a86-5b083b3e7f7a None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:19:09.893713 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a629243f-d13f-4cd0-8a86-5b083b3e7f7a None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:19:09.897157 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be10112-dd86-4b4d-b59f-b8eb585a618f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f4bb9f989e4024a5da28dced646b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:19:09.897778 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be10112-dd86-4b4d-b59f-b8eb585a618f service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:19:09.898059 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be10112-dd86-4b4d-b59f-b8eb585a618f service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:19:09.898334 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be10112-dd86-4b4d-b59f-b8eb585a618f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:19:09.907177 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be10112-dd86-4b4d-b59f-b8eb585a618f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:19:09.909566 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be10112-dd86-4b4d-b59f-b8eb585a618f service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:19:09.921583 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 746/1484] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:19:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:19:11.995790 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d8f22e-d699-44c3-b354-08963aefc396 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:19:11.996361 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d8f22e-d699-44c3-b354-08963aefc396 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:19:11.996458 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d8f22e-d699-44c3-b354-08963aefc396 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:19:11.997768 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 739/1485] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:19:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:19:12.003444 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f324ea7-5f9c-4250-8459-7e58cc759fff None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:19:12.003919 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f324ea7-5f9c-4250-8459-7e58cc759fff None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:19:12.004077 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f324ea7-5f9c-4250-8459-7e58cc759fff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:19:12.022006 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-1f324ea7-5f9c-4250-8459-7e58cc759fff None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:19:12.023189 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-1f324ea7-5f9c-4250-8459-7e58cc759fff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:19:12.030927 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 747/1486] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:19:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:19:12.068163 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d8f22e-d699-44c3-b354-08963aefc396 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:19:12.079244 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d8f22e-d699-44c3-b354-08963aefc396 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:19:12.082669 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98abf52-391a-4293-ba08-3e80e909dc0f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f4bb9f989e4024a5da28dced646b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:19:12.083295 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98abf52-391a-4293-ba08-3e80e909dc0f service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:19:12.083578 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98abf52-391a-4293-ba08-3e80e909dc0f service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:19:12.083883 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98abf52-391a-4293-ba08-3e80e909dc0f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:19:12.092821 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98abf52-391a-4293-ba08-3e80e909dc0f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:19:12.094239 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98abf52-391a-4293-ba08-3e80e909dc0f service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:19:12.103387 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 740/1487] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:19:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:19:14.208934 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc2716e-c3d6-4754-ab1e-bc482ecf4000 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:19:14.209479 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc2716e-c3d6-4754-ab1e-bc482ecf4000 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:19:14.209592 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc2716e-c3d6-4754-ab1e-bc482ecf4000 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:19:14.210870 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 748/1488] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:19:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:19:14.216430 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bf1555-9c34-47c7-a1f8-3f8ec41e3fb3 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:19:14.216853 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bf1555-9c34-47c7-a1f8-3f8ec41e3fb3 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:19:14.216979 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bf1555-9c34-47c7-a1f8-3f8ec41e3fb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:19:14.235125 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-f7bf1555-9c34-47c7-a1f8-3f8ec41e3fb3 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:19:14.236356 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-f7bf1555-9c34-47c7-a1f8-3f8ec41e3fb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:19:14.244113 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 741/1489] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:19:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:19:14.284393 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc2716e-c3d6-4754-ab1e-bc482ecf4000 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:19:14.294822 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc2716e-c3d6-4754-ab1e-bc482ecf4000 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:19:14.298342 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90aa7a85-890b-4d56-a5c6-891b5bad06d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f4bb9f989e4024a5da28dced646b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:19:14.298968 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90aa7a85-890b-4d56-a5c6-891b5bad06d2 service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:19:14.299249 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90aa7a85-890b-4d56-a5c6-891b5bad06d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:19:14.299525 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90aa7a85-890b-4d56-a5c6-891b5bad06d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:19:14.308212 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90aa7a85-890b-4d56-a5c6-891b5bad06d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:19:14.309620 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90aa7a85-890b-4d56-a5c6-891b5bad06d2 service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:19:14.319425 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 749/1490] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:19:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:19:16.690568 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4a4802-dd4f-4aa3-b257-103ad46d69b8 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:19:16.691090 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4a4802-dd4f-4aa3-b257-103ad46d69b8 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:19:16.691207 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4a4802-dd4f-4aa3-b257-103ad46d69b8 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:19:16.692544 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 742/1491] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jun 29 11:19:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:19:16.697697 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b57379d-f3ef-4e8a-a384-20461dfb1faa None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:19:16.698167 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b57379d-f3ef-4e8a-a384-20461dfb1faa None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:19:16.698341 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b57379d-f3ef-4e8a-a384-20461dfb1faa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:19:16.718121 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-0b57379d-f3ef-4e8a-a384-20461dfb1faa None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:19:16.719332 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-0b57379d-f3ef-4e8a-a384-20461dfb1faa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:19:16.727294 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 750/1492] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jun 29 11:19:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:19:16.768361 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4a4802-dd4f-4aa3-b257-103ad46d69b8 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:19:16.778784 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4a4802-dd4f-4aa3-b257-103ad46d69b8 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:19:16.782865 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca1cb6d-5283-404b-ae4f-e026219d2087 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f4bb9f989e4024a5da28dced646b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:19:16.783499 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca1cb6d-5283-404b-ae4f-e026219d2087 service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:19:16.783779 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca1cb6d-5283-404b-ae4f-e026219d2087 service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:19:16.784081 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca1cb6d-5283-404b-ae4f-e026219d2087 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:19:16.794028 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca1cb6d-5283-404b-ae4f-e026219d2087 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:19:16.796111 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca1cb6d-5283-404b-ae4f-e026219d2087 service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:19:16.805300 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 743/1493] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:19:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:19:26.710837 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=33 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:19:26.801232 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=33 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:20:50.636728 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92a812c-c69e-4cbd-8cc6-6516edfce95f None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:20:50.637296 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92a812c-c69e-4cbd-8cc6-6516edfce95f None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:20:50.637296 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92a812c-c69e-4cbd-8cc6-6516edfce95f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:20:50.654736 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-a92a812c-c69e-4cbd-8cc6-6516edfce95f None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:20:50.655974 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-a92a812c-c69e-4cbd-8cc6-6516edfce95f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:20:50.663630 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 751/1494] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:20:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:20:50.671989 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa11e5e-c7e2-4cbe-9ff7-c06a59167c49 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:20:50.672489 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa11e5e-c7e2-4cbe-9ff7-c06a59167c49 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:20:50.672596 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa11e5e-c7e2-4cbe-9ff7-c06a59167c49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:20:50.690807 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-5fa11e5e-c7e2-4cbe-9ff7-c06a59167c49 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:20:50.692040 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-5fa11e5e-c7e2-4cbe-9ff7-c06a59167c49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:20:50.699972 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 744/1495] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:20:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:20:50.704338 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92a812c-c69e-4cbd-8cc6-6516edfce95f None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:20:50.713789 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92a812c-c69e-4cbd-8cc6-6516edfce95f None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:20:50.718537 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e56b1cb-1e53-4910-9d91-f67f643fe82b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:20:50.719183 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e56b1cb-1e53-4910-9d91-f67f643fe82b admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:20:50.719466 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e56b1cb-1e53-4910-9d91-f67f643fe82b admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:20:50.719740 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e56b1cb-1e53-4910-9d91-f67f643fe82b admin admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:20:50.720483 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e56b1cb-1e53-4910-9d91-f67f643fe82b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:20:50.720789 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e56b1cb-1e53-4910-9d91-f67f643fe82b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:20:50.721795 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: WARNING py.warnings [None req-6e56b1cb-1e53-4910-9d91-f67f643fe82b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 29 11:20:50.721795 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: warnings.warn(msg) Jun 29 11:20:50.721795 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: Jun 29 11:20:50.722674 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e56b1cb-1e53-4910-9d91-f67f643fe82b admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:20:50.734431 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 752/1496] 10.222.0.44 () {58 vars in 1152 bytes} [Wed Jun 29 11:20:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:20:50.742134 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa11e5e-c7e2-4cbe-9ff7-c06a59167c49 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:20:50.751616 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa11e5e-c7e2-4cbe-9ff7-c06a59167c49 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:20:50.756193 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7313d74f-6ed6-4431-a530-dea39464e525 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:20:50.756837 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7313d74f-6ed6-4431-a530-dea39464e525 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:20:50.757122 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7313d74f-6ed6-4431-a530-dea39464e525 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:20:50.757398 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7313d74f-6ed6-4431-a530-dea39464e525 admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:20:50.758092 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7313d74f-6ed6-4431-a530-dea39464e525 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:20:50.759869 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: WARNING py.warnings [None req-7313d74f-6ed6-4431-a530-dea39464e525 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 29 11:20:50.759869 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: warnings.warn(msg) Jun 29 11:20:50.759869 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: Jun 29 11:20:50.761047 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7313d74f-6ed6-4431-a530-dea39464e525 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:20:50.862623 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 745/1497] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jun 29 11:20:50 2022] POST /identity/v3/projects => generated 375 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:20:50.866826 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e56b1cb-1e53-4910-9d91-f67f643fe82b admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:20:50.877094 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e56b1cb-1e53-4910-9d91-f67f643fe82b admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:20:50.932457 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3420acd-5567-4ac6-84bf-dbd93a0658dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:20:50.933123 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3420acd-5567-4ac6-84bf-dbd93a0658dc admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:20:50.933404 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3420acd-5567-4ac6-84bf-dbd93a0658dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:20:50.933674 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3420acd-5567-4ac6-84bf-dbd93a0658dc admin admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:20:50.934366 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3420acd-5567-4ac6-84bf-dbd93a0658dc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:20:50.935369 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: WARNING py.warnings [None req-f3420acd-5567-4ac6-84bf-dbd93a0658dc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 29 11:20:50.935369 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: warnings.warn(msg) Jun 29 11:20:50.935369 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: Jun 29 11:20:50.937246 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3420acd-5567-4ac6-84bf-dbd93a0658dc admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:20:50.997236 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 753/1498] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jun 29 11:20:50 2022] POST /identity/v3/users => generated 288 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:20:51.001493 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7313d74f-6ed6-4431-a530-dea39464e525 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:20:51.011696 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7313d74f-6ed6-4431-a530-dea39464e525 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:20:51.016261 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10314241-c9ce-4707-b02d-90b333d9258b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:20:51.016929 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10314241-c9ce-4707-b02d-90b333d9258b admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:20:51.017250 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10314241-c9ce-4707-b02d-90b333d9258b admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:20:51.017558 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10314241-c9ce-4707-b02d-90b333d9258b admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:20:51.018303 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10314241-c9ce-4707-b02d-90b333d9258b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:20:51.019313 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: WARNING py.warnings [None req-10314241-c9ce-4707-b02d-90b333d9258b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 29 11:20:51.019313 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: warnings.warn(msg) Jun 29 11:20:51.019313 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: Jun 29 11:20:51.020286 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10314241-c9ce-4707-b02d-90b333d9258b admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:20:51.033262 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 746/1499] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:20:51 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 11:20:51.036942 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3420acd-5567-4ac6-84bf-dbd93a0658dc admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:20:51.047064 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3420acd-5567-4ac6-84bf-dbd93a0658dc admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:20:51.051468 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2de182-e06d-4c82-8765-9746f76d35da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:20:51.064057 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2de182-e06d-4c82-8765-9746f76d35da admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:20:51.064057 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2de182-e06d-4c82-8765-9746f76d35da admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:20:51.064057 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2de182-e06d-4c82-8765-9746f76d35da admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:20:51.064057 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2de182-e06d-4c82-8765-9746f76d35da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:20:51.064057 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: WARNING py.warnings [None req-0a2de182-e06d-4c82-8765-9746f76d35da admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 29 11:20:51.064057 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: warnings.warn(msg) Jun 29 11:20:51.064057 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: Jun 29 11:20:51.064639 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2de182-e06d-4c82-8765-9746f76d35da admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:20:51.068139 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 754/1500] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:20:51 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 11:20:51.072062 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10314241-c9ce-4707-b02d-90b333d9258b admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:20:51.081680 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10314241-c9ce-4707-b02d-90b333d9258b admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:20:51.086158 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f5b787-8b4a-44a9-b1a7-04c54bb02e4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:20:51.087023 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f5b787-8b4a-44a9-b1a7-04c54bb02e4a admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:20:51.087307 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f5b787-8b4a-44a9-b1a7-04c54bb02e4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:20:51.087583 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f5b787-8b4a-44a9-b1a7-04c54bb02e4a admin admin] PATH_INFO: `/v3/projects/440866d36c8344a9906f290ffff31682/users/b418cc1e156842b0b0a23c244266b728/roles/e1c88eebe911432cab6a07fbeb84653b` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:20:51.102767 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f5b787-8b4a-44a9-b1a7-04c54bb02e4a admin admin] RBAC: Authorizing `identity:create_grant(project_id=440866d36c8344a9906f290ffff31682, user_id=b418cc1e156842b0b0a23c244266b728, role_id=e1c88eebe911432cab6a07fbeb84653b)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:20:51.103789 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: WARNING py.warnings [None req-37f5b787-8b4a-44a9-b1a7-04c54bb02e4a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 29 11:20:51.103789 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: warnings.warn(msg) Jun 29 11:20:51.103789 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: Jun 29 11:20:51.105718 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f5b787-8b4a-44a9-b1a7-04c54bb02e4a admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:20:51.168987 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 747/1501] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jun 29 11:20:51 2022] PUT /identity/v3/projects/440866d36c8344a9906f290ffff31682/users/b418cc1e156842b0b0a23c244266b728/roles/e1c88eebe911432cab6a07fbeb84653b => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 11:20:51.181412 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df60b76-34d5-41cb-ba5c-7f3ece77bb2b None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:20:51.181643 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df60b76-34d5-41cb-ba5c-7f3ece77bb2b None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:20:51.181921 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df60b76-34d5-41cb-ba5c-7f3ece77bb2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:20:51.206976 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-4df60b76-34d5-41cb-ba5c-7f3ece77bb2b None None] MFA Rules not processed for user `b418cc1e156842b0b0a23c244266b728`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:20:51.208191 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-4df60b76-34d5-41cb-ba5c-7f3ece77bb2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:20:51.273781 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 755/1502] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:20:51 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:20:51.286836 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf12bab3-1daf-489d-b6d0-9b113b61abda None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:20:51.287128 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf12bab3-1daf-489d-b6d0-9b113b61abda None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:20:51.287408 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf12bab3-1daf-489d-b6d0-9b113b61abda None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:20:51.288802 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 748/1503] 10.222.0.44 () {54 vars in 963 bytes} [Wed Jun 29 11:20:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:20:51.293394 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac264f12-9121-46b4-ba5e-13adc6359e56 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:20:51.293658 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac264f12-9121-46b4-ba5e-13adc6359e56 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:20:51.293937 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac264f12-9121-46b4-ba5e-13adc6359e56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:20:51.312933 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-ac264f12-9121-46b4-ba5e-13adc6359e56 None None] MFA Rules not processed for user `ff7931b75a924394b1af733febce0834`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:20:51.314085 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-ac264f12-9121-46b4-ba5e-13adc6359e56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:20:51.358126 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:20:51.365218 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 756/1504] 10.222.0.44 () {58 vars in 1060 bytes} [Wed Jun 29 11:20:51 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:20:51.373497 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf12bab3-1daf-489d-b6d0-9b113b61abda None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:20:51.383689 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf12bab3-1daf-489d-b6d0-9b113b61abda None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:20:51.387132 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc0eec8-cc2a-4cb0-aa26-e80c8cbceb0a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff7931b75a924394b1af733febce0834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:20:51.387752 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc0eec8-cc2a-4cb0-aa26-e80c8cbceb0a service cinder] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:20:51.388063 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc0eec8-cc2a-4cb0-aa26-e80c8cbceb0a service cinder] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:20:51.388339 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc0eec8-cc2a-4cb0-aa26-e80c8cbceb0a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:20:51.396208 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc0eec8-cc2a-4cb0-aa26-e80c8cbceb0a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:20:51.397598 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc0eec8-cc2a-4cb0-aa26-e80c8cbceb0a service cinder] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:20:51.406041 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 749/1505] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:20:51 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:20:51.448480 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89727094-55f0-4196-bdec-16f0a04dd7c3 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:20:51.448774 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89727094-55f0-4196-bdec-16f0a04dd7c3 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:20:51.449052 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89727094-55f0-4196-bdec-16f0a04dd7c3 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:20:51.450453 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 757/1506] 10.222.0.44 () {52 vars in 829 bytes} [Wed Jun 29 11:20:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:20:51.454068 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc0eec8-cc2a-4cb0-aa26-e80c8cbceb0a service cinder] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:20:51.519538 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc0eec8-cc2a-4cb0-aa26-e80c8cbceb0a service cinder] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:20:51.524407 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d61842d-56eb-487c-b1b5-c395163c7171 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:20:51.525281 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d61842d-56eb-487c-b1b5-c395163c7171 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:20:51.525691 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d61842d-56eb-487c-b1b5-c395163c7171 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:20:51.525967 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d61842d-56eb-487c-b1b5-c395163c7171 admin admin] PATH_INFO: `/v3/users/b418cc1e156842b0b0a23c244266b728` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:20:51.527141 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d61842d-56eb-487c-b1b5-c395163c7171 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b418cc1e156842b0b0a23c244266b728)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:20:51.528225 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: WARNING py.warnings [None req-0d61842d-56eb-487c-b1b5-c395163c7171 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 29 11:20:51.528225 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: warnings.warn(msg) Jun 29 11:20:51.528225 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: Jun 29 11:20:51.529460 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d61842d-56eb-487c-b1b5-c395163c7171 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:20:51.963823 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-0d61842d-56eb-487c-b1b5-c395163c7171 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b418cc1e156842b0b0a23c244266b728'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:20:51.964489 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-0d61842d-56eb-487c-b1b5-c395163c7171 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b418cc1e156842b0b0a23c244266b728'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:20:52.031585 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-0d61842d-56eb-487c-b1b5-c395163c7171 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b418cc1e156842b0b0a23c244266b728'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:20:52.043747 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-0d61842d-56eb-487c-b1b5-c395163c7171 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b418cc1e156842b0b0a23c244266b728'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:20:52.044500 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-0d61842d-56eb-487c-b1b5-c395163c7171 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b418cc1e156842b0b0a23c244266b728'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:20:52.061003 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:20:52.068672 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 750/1507] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jun 29 11:20:51 2022] DELETE /identity/v3/users/b418cc1e156842b0b0a23c244266b728 => generated 0 bytes in 615 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:20:52.073024 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c03661e-ceee-4afe-986c-0a79bcd1749b None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:20:52.073502 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c03661e-ceee-4afe-986c-0a79bcd1749b None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:20:52.073606 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c03661e-ceee-4afe-986c-0a79bcd1749b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:20:52.091512 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-4c03661e-ceee-4afe-986c-0a79bcd1749b None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:20:52.093375 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-4c03661e-ceee-4afe-986c-0a79bcd1749b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:20:52.153607 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 758/1508] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:20:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:20:52.163646 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d61842d-56eb-487c-b1b5-c395163c7171 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:20:52.165085 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-0d61842d-56eb-487c-b1b5-c395163c7171 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:20:52.225319 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d61842d-56eb-487c-b1b5-c395163c7171 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:20:52.229559 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f70cda2-2e58-47a7-841c-889651832c30 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f4bb9f989e4024a5da28dced646b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:20:52.229559 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f70cda2-2e58-47a7-841c-889651832c30 service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:20:52.229938 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f70cda2-2e58-47a7-841c-889651832c30 service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:20:52.230224 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f70cda2-2e58-47a7-841c-889651832c30 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:20:52.239487 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f70cda2-2e58-47a7-841c-889651832c30 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:20:52.240943 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f70cda2-2e58-47a7-841c-889651832c30 service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:20:52.250448 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 751/1509] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:20:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:20:52.736841 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c03661e-ceee-4afe-986c-0a79bcd1749b None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:20:52.740016 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-4c03661e-ceee-4afe-986c-0a79bcd1749b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:20:52.750706 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c03661e-ceee-4afe-986c-0a79bcd1749b None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:20:52.756137 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e6d9e9-9a27-4146-be36-4a4c7682c728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:20:52.757199 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e6d9e9-9a27-4146-be36-4a4c7682c728 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:20:52.760047 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e6d9e9-9a27-4146-be36-4a4c7682c728 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:20:52.760047 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e6d9e9-9a27-4146-be36-4a4c7682c728 admin admin] PATH_INFO: `/v3/projects/440866d36c8344a9906f290ffff31682` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:20:52.760706 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e6d9e9-9a27-4146-be36-4a4c7682c728 admin admin] RBAC: Authorizing `identity:delete_project(project_id=440866d36c8344a9906f290ffff31682)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:20:52.761829 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: WARNING py.warnings [None req-36e6d9e9-9a27-4146-be36-4a4c7682c728 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 29 11:20:52.761829 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: warnings.warn(msg) Jun 29 11:20:52.761829 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: Jun 29 11:20:52.763275 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e6d9e9-9a27-4146-be36-4a4c7682c728 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:20:52.883582 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-36e6d9e9-9a27-4146-be36-4a4c7682c728 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '440866d36c8344a9906f290ffff31682', 'request_id': 'req-36e6d9e9-9a27-4146-be36-4a4c7682c728'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:20:52.934895 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-36e6d9e9-9a27-4146-be36-4a4c7682c728 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '440866d36c8344a9906f290ffff31682', 'request_id': 'req-36e6d9e9-9a27-4146-be36-4a4c7682c728'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:20:52.951564 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-36e6d9e9-9a27-4146-be36-4a4c7682c728 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '440866d36c8344a9906f290ffff31682', 'request_id': 'req-36e6d9e9-9a27-4146-be36-4a4c7682c728'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:20:52.962665 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-36e6d9e9-9a27-4146-be36-4a4c7682c728 admin admin] The token cache is being invalidate because project 440866d36c8344a9906f290ffff31682 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 11:20:52.963067 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-36e6d9e9-9a27-4146-be36-4a4c7682c728 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:20:52.963713 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-36e6d9e9-9a27-4146-be36-4a4c7682c728 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:20:52.965381 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 759/1510] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jun 29 11:20:52 2022] DELETE /identity/v3/projects/440866d36c8344a9906f290ffff31682 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:21:02.246188 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:DELETE=1,keystone:SELECT=28 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:21:02.959543 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:22:34.405362 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb848508-5016-4871-8e71-e0f0a558790e None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:22:34.405943 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb848508-5016-4871-8e71-e0f0a558790e None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:22:34.405943 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb848508-5016-4871-8e71-e0f0a558790e None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:22:34.407309 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 752/1511] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 29 11:22:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:22:34.412734 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f3c0b3-dccb-4ed7-9ed7-572da470195f None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:22:34.413163 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f3c0b3-dccb-4ed7-9ed7-572da470195f None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:22:34.413294 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f3c0b3-dccb-4ed7-9ed7-572da470195f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:22:34.431795 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-09f3c0b3-dccb-4ed7-9ed7-572da470195f None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:22:34.433026 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-09f3c0b3-dccb-4ed7-9ed7-572da470195f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:22:34.489952 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 760/1512] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 29 11:22:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:22:34.653133 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb848508-5016-4871-8e71-e0f0a558790e None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:22:34.657281 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-eb848508-5016-4871-8e71-e0f0a558790e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:22:34.732677 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb848508-5016-4871-8e71-e0f0a558790e None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:22:34.738266 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586d91c8-52ce-489a-9697-ad8c1e8f6252 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '796aeeff16df4739bddec1f6dee61bfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:22:34.738919 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586d91c8-52ce-489a-9697-ad8c1e8f6252 service nova] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:22:34.739205 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586d91c8-52ce-489a-9697-ad8c1e8f6252 service nova] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:22:34.739485 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586d91c8-52ce-489a-9697-ad8c1e8f6252 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:22:34.747734 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586d91c8-52ce-489a-9697-ad8c1e8f6252 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:22:34.749440 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586d91c8-52ce-489a-9697-ad8c1e8f6252 service nova] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:22:34.759046 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 753/1513] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:22:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:22:35.582274 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581c1041-5683-4845-ac22-8f7e33516dcc None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:22:35.582821 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581c1041-5683-4845-ac22-8f7e33516dcc None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:22:35.582821 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581c1041-5683-4845-ac22-8f7e33516dcc None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:22:35.584658 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 761/1514] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 29 11:22:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:22:35.589333 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2d5278-51ec-40bf-a3bb-532d02e0bfc4 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:22:35.589771 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2d5278-51ec-40bf-a3bb-532d02e0bfc4 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:22:35.589904 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2d5278-51ec-40bf-a3bb-532d02e0bfc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:22:35.608468 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-7c2d5278-51ec-40bf-a3bb-532d02e0bfc4 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:22:35.609653 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-7c2d5278-51ec-40bf-a3bb-532d02e0bfc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:22:35.617531 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 754/1515] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 29 11:22:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:22:35.767770 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581c1041-5683-4845-ac22-8f7e33516dcc None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:22:35.782331 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581c1041-5683-4845-ac22-8f7e33516dcc None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:22:35.787567 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9070889-8cd2-44df-a114-2d4d17ac9764 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '796aeeff16df4739bddec1f6dee61bfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:22:35.788225 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9070889-8cd2-44df-a114-2d4d17ac9764 service nova] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:22:35.788507 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9070889-8cd2-44df-a114-2d4d17ac9764 service nova] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:22:35.788778 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9070889-8cd2-44df-a114-2d4d17ac9764 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:22:35.796707 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9070889-8cd2-44df-a114-2d4d17ac9764 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:22:35.798283 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9070889-8cd2-44df-a114-2d4d17ac9764 service nova] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:22:35.807394 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 762/1516] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:22:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:22:36.853839 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa04a0a-ef66-4f80-b755-4bb2e5bd40fb None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:22:36.854420 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa04a0a-ef66-4f80-b755-4bb2e5bd40fb None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:22:36.854420 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa04a0a-ef66-4f80-b755-4bb2e5bd40fb None None] PATH_INFO: `/v3` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:22:36.855682 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 755/1517] 10.222.0.46 () {54 vars in 925 bytes} [Wed Jun 29 11:22:36 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:22:36.862823 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29826ea9-4265-4192-8070-474607cd7146 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:22:36.863247 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29826ea9-4265-4192-8070-474607cd7146 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:22:36.863411 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29826ea9-4265-4192-8070-474607cd7146 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:22:36.881381 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-29826ea9-4265-4192-8070-474607cd7146 None None] MFA Rules not processed for user `1a0505342b704d02977a5125957c2c9e`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:22:36.882587 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-29826ea9-4265-4192-8070-474607cd7146 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:22:36.951671 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 763/1518] 10.222.0.46 () {58 vars in 1015 bytes} [Wed Jun 29 11:22:36 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:22:45.603398 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=37 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:22:46.143514 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2106c47-7b73-453a-be86-8c1d184ecea6 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:22:46.143723 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2106c47-7b73-453a-be86-8c1d184ecea6 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:22:46.144010 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2106c47-7b73-453a-be86-8c1d184ecea6 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:22:46.145380 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 756/1519] 10.222.0.44 () {54 vars in 960 bytes} [Wed Jun 29 11:22:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:22:46.150382 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257fe0b0-dc82-4701-a13c-582c9b7ed328 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:22:46.150841 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257fe0b0-dc82-4701-a13c-582c9b7ed328 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:22:46.150950 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257fe0b0-dc82-4701-a13c-582c9b7ed328 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:22:46.168759 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-257fe0b0-dc82-4701-a13c-582c9b7ed328 None None] MFA Rules not processed for user `1a0505342b704d02977a5125957c2c9e`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:22:46.169936 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-257fe0b0-dc82-4701-a13c-582c9b7ed328 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:22:46.178770 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 764/1520] 10.222.0.44 () {58 vars in 1057 bytes} [Wed Jun 29 11:22:46 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:22:46.186164 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2106c47-7b73-453a-be86-8c1d184ecea6 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:22:46.200063 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2106c47-7b73-453a-be86-8c1d184ecea6 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:22:46.204908 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecef225-d1d0-41ba-ad2f-22c2e49e1199 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a0505342b704d02977a5125957c2c9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:22:46.205549 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecef225-d1d0-41ba-ad2f-22c2e49e1199 service placement] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:22:46.205857 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecef225-d1d0-41ba-ad2f-22c2e49e1199 service placement] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:22:46.206130 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecef225-d1d0-41ba-ad2f-22c2e49e1199 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:22:46.214849 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecef225-d1d0-41ba-ad2f-22c2e49e1199 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:22:46.216454 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecef225-d1d0-41ba-ad2f-22c2e49e1199 service placement] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:22:46.227085 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 757/1521] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:22:46 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:22:47.048879 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc583e32-3626-4960-be64-1f6f39ad91dc None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:22:47.049168 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc583e32-3626-4960-be64-1f6f39ad91dc None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:22:47.049451 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc583e32-3626-4960-be64-1f6f39ad91dc None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:22:47.050944 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 765/1522] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 29 11:22:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 29 11:22:47.055773 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97920b17-7a66-4821-ba4a-4a96f8065b9a None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:22:47.059261 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97920b17-7a66-4821-ba4a-4a96f8065b9a None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:22:47.059261 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97920b17-7a66-4821-ba4a-4a96f8065b9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:22:47.074628 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-97920b17-7a66-4821-ba4a-4a96f8065b9a None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:22:47.075815 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-97920b17-7a66-4821-ba4a-4a96f8065b9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:22:47.083725 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 758/1523] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 29 11:22:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:22:47.239621 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc583e32-3626-4960-be64-1f6f39ad91dc None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:22:47.253839 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc583e32-3626-4960-be64-1f6f39ad91dc None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:22:47.259691 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d966f3-a7d7-4dd2-a56a-4fb89c95230d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '796aeeff16df4739bddec1f6dee61bfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:22:47.260345 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d966f3-a7d7-4dd2-a56a-4fb89c95230d service nova] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:22:47.260622 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d966f3-a7d7-4dd2-a56a-4fb89c95230d service nova] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:22:47.260893 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d966f3-a7d7-4dd2-a56a-4fb89c95230d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:22:47.269009 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d966f3-a7d7-4dd2-a56a-4fb89c95230d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:22:47.270648 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d966f3-a7d7-4dd2-a56a-4fb89c95230d service nova] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:22:47.280879 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 766/1524] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:22:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:22:48.087372 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed981eb-9d17-4a22-b588-ce0e544c760f None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:22:48.087927 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed981eb-9d17-4a22-b588-ce0e544c760f None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:22:48.088040 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed981eb-9d17-4a22-b588-ce0e544c760f None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:22:48.089337 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 759/1525] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 29 11:22:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:22:48.094215 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a99c20-75a2-45d0-b7ab-c04f945dbe73 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:22:48.094498 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a99c20-75a2-45d0-b7ab-c04f945dbe73 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:22:48.094770 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a99c20-75a2-45d0-b7ab-c04f945dbe73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:22:48.112678 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-54a99c20-75a2-45d0-b7ab-c04f945dbe73 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:22:48.113862 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-54a99c20-75a2-45d0-b7ab-c04f945dbe73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:22:48.121628 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 767/1526] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 29 11:22:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:22:48.271300 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed981eb-9d17-4a22-b588-ce0e544c760f None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:22:48.285350 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed981eb-9d17-4a22-b588-ce0e544c760f None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:22:48.290651 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7f771b-5b05-44cf-b649-f21ee4e924ab service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '796aeeff16df4739bddec1f6dee61bfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:22:48.291278 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7f771b-5b05-44cf-b649-f21ee4e924ab service nova] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:22:48.291560 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7f771b-5b05-44cf-b649-f21ee4e924ab service nova] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:22:48.291852 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7f771b-5b05-44cf-b649-f21ee4e924ab service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:22:48.299732 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7f771b-5b05-44cf-b649-f21ee4e924ab service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:22:48.301392 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7f771b-5b05-44cf-b649-f21ee4e924ab service nova] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:22:48.311266 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 760/1527] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:22:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:22:52.288646 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc745fd6-de0a-431f-885a-d8ed98cdf9b1 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:22:52.289195 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc745fd6-de0a-431f-885a-d8ed98cdf9b1 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:22:52.289195 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc745fd6-de0a-431f-885a-d8ed98cdf9b1 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:22:52.290542 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 768/1528] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 29 11:22:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:22:52.296068 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322d6da0-7481-4977-ad9a-cf7fa914e381 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:22:52.296474 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322d6da0-7481-4977-ad9a-cf7fa914e381 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:22:52.296634 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322d6da0-7481-4977-ad9a-cf7fa914e381 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:22:52.315075 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-322d6da0-7481-4977-ad9a-cf7fa914e381 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:22:52.316354 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-322d6da0-7481-4977-ad9a-cf7fa914e381 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:22:52.326703 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 761/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 29 11:22:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:22:52.366572 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc745fd6-de0a-431f-885a-d8ed98cdf9b1 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:22:52.368288 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-fc745fd6-de0a-431f-885a-d8ed98cdf9b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:22:52.424031 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc745fd6-de0a-431f-885a-d8ed98cdf9b1 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:22:52.427485 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fbc595-4393-481c-899e-b372ade8019b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f4bb9f989e4024a5da28dced646b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:22:52.428764 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fbc595-4393-481c-899e-b372ade8019b service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:22:52.429050 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fbc595-4393-481c-899e-b372ade8019b service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:22:52.429328 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fbc595-4393-481c-899e-b372ade8019b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:22:52.438055 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fbc595-4393-481c-899e-b372ade8019b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:22:52.439525 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fbc595-4393-481c-899e-b372ade8019b service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:22:52.444424 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:22:52.449130 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 769/1530] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:22:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:22:54.467049 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4812e7fb-9521-44fd-8af8-913fca14dc92 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:22:54.467873 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4812e7fb-9521-44fd-8af8-913fca14dc92 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:22:54.467873 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4812e7fb-9521-44fd-8af8-913fca14dc92 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:22:54.469260 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 762/1531] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 29 11:22:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:22:54.474987 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d355040-c317-4e0f-b013-e157a7b49ed2 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:22:54.475404 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d355040-c317-4e0f-b013-e157a7b49ed2 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:22:54.475537 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d355040-c317-4e0f-b013-e157a7b49ed2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:22:54.493881 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-4d355040-c317-4e0f-b013-e157a7b49ed2 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:22:54.495101 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-4d355040-c317-4e0f-b013-e157a7b49ed2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:22:54.502734 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 770/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 29 11:22:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:22:54.538413 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4812e7fb-9521-44fd-8af8-913fca14dc92 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:22:54.548975 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4812e7fb-9521-44fd-8af8-913fca14dc92 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:22:54.552402 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fb9091-5d23-48db-9e97-718b35abb0b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f4bb9f989e4024a5da28dced646b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:22:54.553628 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fb9091-5d23-48db-9e97-718b35abb0b4 service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:22:54.553911 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fb9091-5d23-48db-9e97-718b35abb0b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:22:54.554187 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fb9091-5d23-48db-9e97-718b35abb0b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:22:54.562881 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fb9091-5d23-48db-9e97-718b35abb0b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:22:54.564344 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fb9091-5d23-48db-9e97-718b35abb0b4 service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:22:54.574125 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 763/1533] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:22:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:22:58.068115 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca453c7-43d5-499a-954a-30078e8308c6 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:22:58.068681 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca453c7-43d5-499a-954a-30078e8308c6 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:22:58.068681 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca453c7-43d5-499a-954a-30078e8308c6 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:22:58.070040 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 771/1534] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 29 11:22:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:22:58.075068 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d800fc3-90f3-42e4-8872-792df7913350 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:22:58.075475 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d800fc3-90f3-42e4-8872-792df7913350 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:22:58.075626 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d800fc3-90f3-42e4-8872-792df7913350 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:22:58.094038 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-7d800fc3-90f3-42e4-8872-792df7913350 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:22:58.095232 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-7d800fc3-90f3-42e4-8872-792df7913350 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:22:58.103520 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 764/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 29 11:22:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:22:58.407098 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca453c7-43d5-499a-954a-30078e8308c6 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:22:58.408764 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-8ca453c7-43d5-499a-954a-30078e8308c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:22:58.466134 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca453c7-43d5-499a-954a-30078e8308c6 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:22:58.469924 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf118da-2fa7-4fbf-916e-3f0548c92327 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff7931b75a924394b1af733febce0834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:22:58.470567 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf118da-2fa7-4fbf-916e-3f0548c92327 service cinder] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:22:58.470849 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf118da-2fa7-4fbf-916e-3f0548c92327 service cinder] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:22:58.471126 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf118da-2fa7-4fbf-916e-3f0548c92327 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:22:58.479917 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf118da-2fa7-4fbf-916e-3f0548c92327 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:22:58.481345 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf118da-2fa7-4fbf-916e-3f0548c92327 service cinder] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:22:58.490629 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 772/1536] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:22:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:23:00.548994 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da52399d-cf57-4f73-a792-8447b224ae02 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:23:00.549527 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da52399d-cf57-4f73-a792-8447b224ae02 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:23:00.549527 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da52399d-cf57-4f73-a792-8447b224ae02 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:23:00.550908 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 765/1537] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 29 11:23:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:23:00.555979 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf5949a-440a-4683-ac09-92eb5f5c3127 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:23:00.556400 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf5949a-440a-4683-ac09-92eb5f5c3127 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:23:00.556574 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf5949a-440a-4683-ac09-92eb5f5c3127 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:23:00.574910 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-5cf5949a-440a-4683-ac09-92eb5f5c3127 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:23:00.576171 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-5cf5949a-440a-4683-ac09-92eb5f5c3127 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:23:00.584481 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 773/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 29 11:23:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:23:00.888629 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da52399d-cf57-4f73-a792-8447b224ae02 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:23:00.890310 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-da52399d-cf57-4f73-a792-8447b224ae02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:23:00.903032 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da52399d-cf57-4f73-a792-8447b224ae02 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:23:00.906485 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbcfe56-f10f-43fe-9cde-542e2f571f31 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff7931b75a924394b1af733febce0834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:23:00.907112 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbcfe56-f10f-43fe-9cde-542e2f571f31 service cinder] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:23:00.907396 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbcfe56-f10f-43fe-9cde-542e2f571f31 service cinder] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:23:00.907672 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbcfe56-f10f-43fe-9cde-542e2f571f31 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:23:00.917134 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbcfe56-f10f-43fe-9cde-542e2f571f31 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:23:00.918550 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbcfe56-f10f-43fe-9cde-542e2f571f31 service cinder] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:23:00.929172 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 766/1539] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:23:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:23:10.569780 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=32 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:23:10.923738 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=33 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:24:30.410041 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea81bc9-7c41-44c7-8aca-e834ce5417f4 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:30.413353 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea81bc9-7c41-44c7-8aca-e834ce5417f4 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:30.413353 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea81bc9-7c41-44c7-8aca-e834ce5417f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:30.442375 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-4ea81bc9-7c41-44c7-8aca-e834ce5417f4 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:24:30.443973 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-4ea81bc9-7c41-44c7-8aca-e834ce5417f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:24:30.459890 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 774/1540] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:24:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:24:30.469712 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cf56d1-70c7-4d7b-9a09-c0c2231e0dbf None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:30.471319 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cf56d1-70c7-4d7b-9a09-c0c2231e0dbf None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:30.471319 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cf56d1-70c7-4d7b-9a09-c0c2231e0dbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:30.495220 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-d2cf56d1-70c7-4d7b-9a09-c0c2231e0dbf None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:24:30.496938 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-d2cf56d1-70c7-4d7b-9a09-c0c2231e0dbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:24:30.508013 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 767/1541] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:24:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:24:30.511863 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea81bc9-7c41-44c7-8aca-e834ce5417f4 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:24:30.529257 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea81bc9-7c41-44c7-8aca-e834ce5417f4 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:24:30.537888 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e94949-1dcf-48fd-9135-9598fa4f7bbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:24:30.538565 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e94949-1dcf-48fd-9135-9598fa4f7bbe admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:30.538851 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e94949-1dcf-48fd-9135-9598fa4f7bbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:30.539131 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e94949-1dcf-48fd-9135-9598fa4f7bbe admin admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:30.539943 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e94949-1dcf-48fd-9135-9598fa4f7bbe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:24:30.540251 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e94949-1dcf-48fd-9135-9598fa4f7bbe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:24:30.541878 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e94949-1dcf-48fd-9135-9598fa4f7bbe admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:24:30.564199 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 775/1542] 10.222.0.44 () {58 vars in 1152 bytes} [Wed Jun 29 11:24:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:24:30.573284 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cf56d1-70c7-4d7b-9a09-c0c2231e0dbf None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:24:30.583121 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cf56d1-70c7-4d7b-9a09-c0c2231e0dbf None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:24:30.592858 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e37f6e-06f8-427d-8c86-6a5ee94554a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:24:30.594113 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e37f6e-06f8-427d-8c86-6a5ee94554a9 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:30.594617 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e37f6e-06f8-427d-8c86-6a5ee94554a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:30.595114 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e37f6e-06f8-427d-8c86-6a5ee94554a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:30.596053 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e37f6e-06f8-427d-8c86-6a5ee94554a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:24:30.598175 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e37f6e-06f8-427d-8c86-6a5ee94554a9 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:24:30.663521 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4474357a-5efb-406d-bd37-f79117f8fe2a None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:30.663894 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4474357a-5efb-406d-bd37-f79117f8fe2a None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:30.664194 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4474357a-5efb-406d-bd37-f79117f8fe2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:30.697154 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-4474357a-5efb-406d-bd37-f79117f8fe2a None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:24:30.703952 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-4474357a-5efb-406d-bd37-f79117f8fe2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:24:30.704084 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 768/1543] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jun 29 11:24:30 2022] POST /identity/v3/projects => generated 379 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:24:30.704607 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f7dd52-8901-4cb4-b9e4-b643cf2e2622 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:30.705304 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f7dd52-8901-4cb4-b9e4-b643cf2e2622 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:30.705775 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f7dd52-8901-4cb4-b9e4-b643cf2e2622 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:30.730184 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-22f7dd52-8901-4cb4-b9e4-b643cf2e2622 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:24:30.732586 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-22f7dd52-8901-4cb4-b9e4-b643cf2e2622 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:24:30.797286 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 769/1544] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:24:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:24:30.799204 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f779fcc7-e169-4ef3-a012-a140ee788b69 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:30.799759 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f779fcc7-e169-4ef3-a012-a140ee788b69 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:30.800276 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f779fcc7-e169-4ef3-a012-a140ee788b69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:30.819904 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-f779fcc7-e169-4ef3-a012-a140ee788b69 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:24:30.822296 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-f779fcc7-e169-4ef3-a012-a140ee788b69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:24:30.824867 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 776/1545] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:24:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:24:30.825789 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4474357a-5efb-406d-bd37-f79117f8fe2a None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:24:30.831889 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 770/1546] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:24:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:24:30.833528 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e62c13-ebdf-4925-b55d-5e5e739aa1c1 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:30.834036 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e62c13-ebdf-4925-b55d-5e5e739aa1c1 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:30.834532 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e62c13-ebdf-4925-b55d-5e5e739aa1c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:30.842769 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4474357a-5efb-406d-bd37-f79117f8fe2a None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:24:30.847361 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae27e0bc-62e4-418d-8f71-34fad7e91105 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:24:30.848081 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae27e0bc-62e4-418d-8f71-34fad7e91105 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:30.848367 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae27e0bc-62e4-418d-8f71-34fad7e91105 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:30.848644 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae27e0bc-62e4-418d-8f71-34fad7e91105 admin admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:30.849679 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae27e0bc-62e4-418d-8f71-34fad7e91105 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:24:30.851425 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae27e0bc-62e4-418d-8f71-34fad7e91105 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:24:30.855670 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-95e62c13-ebdf-4925-b55d-5e5e739aa1c1 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:24:30.857142 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-95e62c13-ebdf-4925-b55d-5e5e739aa1c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:24:30.866577 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 771/1547] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:24:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:24:30.868798 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb9ccb3-1e9c-4133-9c31-b211baca0a8f None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:30.869308 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb9ccb3-1e9c-4133-9c31-b211baca0a8f None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:30.869803 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb9ccb3-1e9c-4133-9c31-b211baca0a8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:30.890384 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-abb9ccb3-1e9c-4133-9c31-b211baca0a8f None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:24:30.891910 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-abb9ccb3-1e9c-4133-9c31-b211baca0a8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:24:30.901767 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 772/1548] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:24:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:24:30.903248 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20ba979-aa2e-4f84-a9cf-752ad0595f53 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:30.903539 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20ba979-aa2e-4f84-a9cf-752ad0595f53 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:30.903815 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20ba979-aa2e-4f84-a9cf-752ad0595f53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:30.925319 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-d20ba979-aa2e-4f84-a9cf-752ad0595f53 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:24:30.926517 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-d20ba979-aa2e-4f84-a9cf-752ad0595f53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:24:30.934793 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 773/1549] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:24:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:24:30.936223 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a6f198-fe4c-484a-a608-aac5ad7acece None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:30.936502 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a6f198-fe4c-484a-a608-aac5ad7acece None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:30.936774 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a6f198-fe4c-484a-a608-aac5ad7acece None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:30.954314 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-39a6f198-fe4c-484a-a608-aac5ad7acece None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:24:30.954504 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 777/1550] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jun 29 11:24:30 2022] POST /identity/v3/users => generated 290 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:24:30.955558 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-39a6f198-fe4c-484a-a608-aac5ad7acece None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:24:30.955798 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b1444d-bdb9-47fa-8358-ee5d414c56c6 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:30.956268 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b1444d-bdb9-47fa-8358-ee5d414c56c6 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:30.956653 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b1444d-bdb9-47fa-8358-ee5d414c56c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:30.964870 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 774/1551] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:24:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:24:30.967380 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd20c26d-a78a-46ad-8f37-1f59d27bf5a5 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:30.967380 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd20c26d-a78a-46ad-8f37-1f59d27bf5a5 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:30.967380 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd20c26d-a78a-46ad-8f37-1f59d27bf5a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:30.976733 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-39b1444d-bdb9-47fa-8358-ee5d414c56c6 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:24:30.977964 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-39b1444d-bdb9-47fa-8358-ee5d414c56c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:24:30.984599 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-bd20c26d-a78a-46ad-8f37-1f59d27bf5a5 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:24:30.985788 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-bd20c26d-a78a-46ad-8f37-1f59d27bf5a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:24:30.985927 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 778/1552] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:24:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:24:30.986685 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b1444d-bdb9-47fa-8358-ee5d414c56c6 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:24:30.994384 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 775/1553] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:24:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:24:30.995507 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadf60cd-450d-4919-ab3c-fbfbc674cbbf None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:30.995805 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadf60cd-450d-4919-ab3c-fbfbc674cbbf None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:30.996118 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadf60cd-450d-4919-ab3c-fbfbc674cbbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:31.000055 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b1444d-bdb9-47fa-8358-ee5d414c56c6 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:24:31.004451 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6431d320-227b-41fa-a53d-afc6c66c9bdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:24:31.005097 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6431d320-227b-41fa-a53d-afc6c66c9bdb admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:31.005369 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6431d320-227b-41fa-a53d-afc6c66c9bdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:31.005646 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6431d320-227b-41fa-a53d-afc6c66c9bdb admin admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:31.006361 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6431d320-227b-41fa-a53d-afc6c66c9bdb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:24:31.006664 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6431d320-227b-41fa-a53d-afc6c66c9bdb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:24:31.008853 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6431d320-227b-41fa-a53d-afc6c66c9bdb admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:24:31.013800 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-aadf60cd-450d-4919-ab3c-fbfbc674cbbf None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:24:31.014990 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-aadf60cd-450d-4919-ab3c-fbfbc674cbbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:24:31.020607 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 779/1554] 10.222.0.44 () {58 vars in 1152 bytes} [Wed Jun 29 11:24:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:24:31.020607 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e53412-2fc8-4aac-bebd-21e1ee8ef0df None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:31.020607 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e53412-2fc8-4aac-bebd-21e1ee8ef0df None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:31.020607 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e53412-2fc8-4aac-bebd-21e1ee8ef0df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:31.030780 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 776/1555] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:24:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:24:31.031581 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadf60cd-450d-4919-ab3c-fbfbc674cbbf None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:24:31.040450 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-b5e53412-2fc8-4aac-bebd-21e1ee8ef0df None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:24:31.041674 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-b5e53412-2fc8-4aac-bebd-21e1ee8ef0df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:24:31.043580 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadf60cd-450d-4919-ab3c-fbfbc674cbbf None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:24:31.048086 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe554b8-b25b-4cb0-9d40-40643b23aafe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:24:31.048732 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe554b8-b25b-4cb0-9d40-40643b23aafe admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:31.059873 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe554b8-b25b-4cb0-9d40-40643b23aafe admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:31.059873 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe554b8-b25b-4cb0-9d40-40643b23aafe admin admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:31.060098 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 780/1556] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:24:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:24:31.060205 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe554b8-b25b-4cb0-9d40-40643b23aafe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:24:31.060205 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe554b8-b25b-4cb0-9d40-40643b23aafe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:24:31.060386 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e53412-2fc8-4aac-bebd-21e1ee8ef0df None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:24:31.060474 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: WARNING py.warnings [None req-fbe554b8-b25b-4cb0-9d40-40643b23aafe admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 29 11:24:31.060474 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: warnings.warn(msg) Jun 29 11:24:31.060474 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: Jun 29 11:24:31.060474 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe554b8-b25b-4cb0-9d40-40643b23aafe admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:24:31.060768 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e53412-2fc8-4aac-bebd-21e1ee8ef0df None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:24:31.072413 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 777/1557] 10.222.0.44 () {58 vars in 1152 bytes} [Wed Jun 29 11:24:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:24:31.072588 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b964cd-0fbd-4530-a934-4e7850cca832 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:24:31.072588 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b964cd-0fbd-4530-a934-4e7850cca832 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:31.072839 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7eb3c34-5bd6-42ea-8ed4-3f85f7d9b7cb None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:31.072947 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b964cd-0fbd-4530-a934-4e7850cca832 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:31.073047 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7eb3c34-5bd6-42ea-8ed4-3f85f7d9b7cb None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:31.073149 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b964cd-0fbd-4530-a934-4e7850cca832 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:31.073254 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7eb3c34-5bd6-42ea-8ed4-3f85f7d9b7cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:31.073359 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b964cd-0fbd-4530-a934-4e7850cca832 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:24:31.073359 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b964cd-0fbd-4530-a934-4e7850cca832 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:24:31.082670 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 781/1558] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:24:31 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 11:24:31.090507 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b964cd-0fbd-4530-a934-4e7850cca832 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:24:31.090641 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-f7eb3c34-5bd6-42ea-8ed4-3f85f7d9b7cb None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:24:31.090641 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-f7eb3c34-5bd6-42ea-8ed4-3f85f7d9b7cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:24:31.095114 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b964cd-0fbd-4530-a934-4e7850cca832 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:24:31.095247 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 778/1559] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:24:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:24:31.096061 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7eb3c34-5bd6-42ea-8ed4-3f85f7d9b7cb None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:24:31.099971 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190c82f7-0996-4854-aad6-d79d46fdd1ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:24:31.100610 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190c82f7-0996-4854-aad6-d79d46fdd1ea admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:31.100893 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190c82f7-0996-4854-aad6-d79d46fdd1ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:31.101176 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190c82f7-0996-4854-aad6-d79d46fdd1ea admin admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:31.102395 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190c82f7-0996-4854-aad6-d79d46fdd1ea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:24:31.102708 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190c82f7-0996-4854-aad6-d79d46fdd1ea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:24:31.104312 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190c82f7-0996-4854-aad6-d79d46fdd1ea admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:24:31.105787 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7eb3c34-5bd6-42ea-8ed4-3f85f7d9b7cb None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:24:31.110649 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d4df66-874f-4370-b93c-886e057df28e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:24:31.111312 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d4df66-874f-4370-b93c-886e057df28e admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:31.111598 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d4df66-874f-4370-b93c-886e057df28e admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:31.111912 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d4df66-874f-4370-b93c-886e057df28e admin admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:31.112665 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d4df66-874f-4370-b93c-886e057df28e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:24:31.112977 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d4df66-874f-4370-b93c-886e057df28e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:24:31.114584 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d4df66-874f-4370-b93c-886e057df28e admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:24:31.116212 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 782/1560] 10.222.0.44 () {58 vars in 1152 bytes} [Wed Jun 29 11:24:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:24:31.117273 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190c82f7-0996-4854-aad6-d79d46fdd1ea admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:24:31.127475 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 779/1561] 10.222.0.44 () {58 vars in 1152 bytes} [Wed Jun 29 11:24:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:24:31.128750 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6e0c57-456f-496b-8a2c-02b10a9b79ba None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:31.129044 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6e0c57-456f-496b-8a2c-02b10a9b79ba None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:31.129322 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6e0c57-456f-496b-8a2c-02b10a9b79ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:31.133195 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190c82f7-0996-4854-aad6-d79d46fdd1ea admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:24:31.137880 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f069b9f6-9de5-42da-81a5-35baaa3c9117 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:24:31.138528 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f069b9f6-9de5-42da-81a5-35baaa3c9117 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:31.138811 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f069b9f6-9de5-42da-81a5-35baaa3c9117 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:31.139089 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f069b9f6-9de5-42da-81a5-35baaa3c9117 admin admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:31.139813 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f069b9f6-9de5-42da-81a5-35baaa3c9117 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:24:31.140867 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: WARNING py.warnings [None req-f069b9f6-9de5-42da-81a5-35baaa3c9117 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 29 11:24:31.140867 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: warnings.warn(msg) Jun 29 11:24:31.140867 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: Jun 29 11:24:31.142061 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f069b9f6-9de5-42da-81a5-35baaa3c9117 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:24:31.146726 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-eb6e0c57-456f-496b-8a2c-02b10a9b79ba None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:24:31.147968 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-eb6e0c57-456f-496b-8a2c-02b10a9b79ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:24:31.159255 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 780/1562] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:24:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:24:31.159255 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1648ee-49ff-45f3-87be-0fb81c73b428 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:31.159255 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1648ee-49ff-45f3-87be-0fb81c73b428 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:31.159255 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1648ee-49ff-45f3-87be-0fb81c73b428 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:31.175442 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-0c1648ee-49ff-45f3-87be-0fb81c73b428 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:24:31.176659 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-0c1648ee-49ff-45f3-87be-0fb81c73b428 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:24:31.185020 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 781/1563] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:24:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:24:31.185753 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1648ee-49ff-45f3-87be-0fb81c73b428 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:24:31.194702 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1648ee-49ff-45f3-87be-0fb81c73b428 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:24:31.199224 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac31a33-5e79-4564-b533-472afece71e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:24:31.199852 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac31a33-5e79-4564-b533-472afece71e1 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:31.200153 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac31a33-5e79-4564-b533-472afece71e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:31.200430 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac31a33-5e79-4564-b533-472afece71e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:31.201098 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac31a33-5e79-4564-b533-472afece71e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:24:31.202956 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac31a33-5e79-4564-b533-472afece71e1 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:24:31.243466 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 783/1564] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jun 29 11:24:31 2022] POST /identity/v3/projects => generated 385 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:24:31.243922 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f069b9f6-9de5-42da-81a5-35baaa3c9117 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:24:31.296316 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:24:31.307060 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 782/1565] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jun 29 11:24:31 2022] POST /identity/v3/projects => generated 373 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:24:31.309630 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac31a33-5e79-4564-b533-472afece71e1 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:24:31.311653 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f069b9f6-9de5-42da-81a5-35baaa3c9117 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:24:31.337284 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:24:31.376077 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac31a33-5e79-4564-b533-472afece71e1 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:24:31.377005 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba4c3f4-f3d6-4b5c-8818-041c6778f1bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:24:31.377685 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba4c3f4-f3d6-4b5c-8818-041c6778f1bc admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:31.378089 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba4c3f4-f3d6-4b5c-8818-041c6778f1bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:31.378475 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba4c3f4-f3d6-4b5c-8818-041c6778f1bc admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:31.379213 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba4c3f4-f3d6-4b5c-8818-041c6778f1bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:24:31.380889 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba4c3f4-f3d6-4b5c-8818-041c6778f1bc admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:24:31.381033 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cd03c1-6d16-4789-b467-0c538722156b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:24:31.381642 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cd03c1-6d16-4789-b467-0c538722156b admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:31.381941 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cd03c1-6d16-4789-b467-0c538722156b admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:31.382443 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cd03c1-6d16-4789-b467-0c538722156b admin admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:31.383367 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cd03c1-6d16-4789-b467-0c538722156b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:24:31.383803 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cd03c1-6d16-4789-b467-0c538722156b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:24:31.385905 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cd03c1-6d16-4789-b467-0c538722156b admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:24:31.390957 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 784/1566] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:24:31 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 11:24:31.391764 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba4c3f4-f3d6-4b5c-8818-041c6778f1bc admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:24:31.398030 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 783/1567] 10.222.0.44 () {58 vars in 1152 bytes} [Wed Jun 29 11:24:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:24:31.399502 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cd03c1-6d16-4789-b467-0c538722156b admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:24:31.412791 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba4c3f4-f3d6-4b5c-8818-041c6778f1bc admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:24:31.416654 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cd03c1-6d16-4789-b467-0c538722156b admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:24:31.420467 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41ec934-44d9-43b6-9b54-3216192281cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:24:31.420467 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41ec934-44d9-43b6-9b54-3216192281cc admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:31.420467 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41ec934-44d9-43b6-9b54-3216192281cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:31.420467 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41ec934-44d9-43b6-9b54-3216192281cc admin admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:31.420908 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41ec934-44d9-43b6-9b54-3216192281cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:24:31.422899 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41ec934-44d9-43b6-9b54-3216192281cc admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:24:31.427115 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0884d29-d092-4148-be30-cf8b80f0c790 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:24:31.427955 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0884d29-d092-4148-be30-cf8b80f0c790 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:31.428290 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0884d29-d092-4148-be30-cf8b80f0c790 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:31.428611 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0884d29-d092-4148-be30-cf8b80f0c790 admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:31.429361 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0884d29-d092-4148-be30-cf8b80f0c790 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:24:31.432127 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0884d29-d092-4148-be30-cf8b80f0c790 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:24:31.540663 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 785/1568] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jun 29 11:24:31 2022] POST /identity/v3/projects => generated 375 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:24:31.541820 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41ec934-44d9-43b6-9b54-3216192281cc admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:24:31.584760 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 784/1569] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jun 29 11:24:31 2022] POST /identity/v3/projects => generated 377 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:24:31.585643 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0884d29-d092-4148-be30-cf8b80f0c790 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:24:31.608237 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41ec934-44d9-43b6-9b54-3216192281cc admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:24:31.650813 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0884d29-d092-4148-be30-cf8b80f0c790 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:24:31.655644 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49447c3a-af2c-4584-8110-e232f4f89859 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:24:31.656429 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49447c3a-af2c-4584-8110-e232f4f89859 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:31.656712 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49447c3a-af2c-4584-8110-e232f4f89859 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:31.656994 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49447c3a-af2c-4584-8110-e232f4f89859 admin admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:31.657713 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49447c3a-af2c-4584-8110-e232f4f89859 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:24:31.658022 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49447c3a-af2c-4584-8110-e232f4f89859 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:24:31.659697 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49447c3a-af2c-4584-8110-e232f4f89859 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:24:31.662805 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8759d401-39d7-4309-afc2-76b9111b78eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:24:31.663498 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8759d401-39d7-4309-afc2-76b9111b78eb admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:31.663812 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8759d401-39d7-4309-afc2-76b9111b78eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:31.664122 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8759d401-39d7-4309-afc2-76b9111b78eb admin admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:31.665452 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8759d401-39d7-4309-afc2-76b9111b78eb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:24:31.665759 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8759d401-39d7-4309-afc2-76b9111b78eb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:24:31.667367 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8759d401-39d7-4309-afc2-76b9111b78eb admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:24:31.674362 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 785/1570] 10.222.0.44 () {58 vars in 1152 bytes} [Wed Jun 29 11:24:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:24:31.674362 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49447c3a-af2c-4584-8110-e232f4f89859 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:24:31.691903 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 786/1571] 10.222.0.44 () {58 vars in 1152 bytes} [Wed Jun 29 11:24:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:24:31.691903 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8759d401-39d7-4309-afc2-76b9111b78eb admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:24:31.692179 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49447c3a-af2c-4584-8110-e232f4f89859 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:24:31.692179 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa10b749-dec4-43d2-875d-2f70fecb0705 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:24:31.692179 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa10b749-dec4-43d2-875d-2f70fecb0705 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:31.692179 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa10b749-dec4-43d2-875d-2f70fecb0705 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:31.692469 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa10b749-dec4-43d2-875d-2f70fecb0705 admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:31.692646 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa10b749-dec4-43d2-875d-2f70fecb0705 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:24:31.693655 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: WARNING py.warnings [None req-fa10b749-dec4-43d2-875d-2f70fecb0705 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 29 11:24:31.693655 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: warnings.warn(msg) Jun 29 11:24:31.693655 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: Jun 29 11:24:31.694856 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa10b749-dec4-43d2-875d-2f70fecb0705 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:24:31.697700 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8759d401-39d7-4309-afc2-76b9111b78eb admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:24:31.702206 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985eb15a-e83a-44bc-89aa-44e6d4f9a239 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:24:31.702848 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985eb15a-e83a-44bc-89aa-44e6d4f9a239 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:31.703130 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985eb15a-e83a-44bc-89aa-44e6d4f9a239 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:31.703402 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985eb15a-e83a-44bc-89aa-44e6d4f9a239 admin admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:31.704106 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985eb15a-e83a-44bc-89aa-44e6d4f9a239 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:24:31.706019 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985eb15a-e83a-44bc-89aa-44e6d4f9a239 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:24:31.715879 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:24:31.798654 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 786/1572] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jun 29 11:24:31 2022] POST /identity/v3/users => generated 293 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:24:31.799586 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa10b749-dec4-43d2-875d-2f70fecb0705 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:24:31.808866 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa10b749-dec4-43d2-875d-2f70fecb0705 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:24:31.813730 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702dea21-df40-4698-90e5-7e16f65d77cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:24:31.814617 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702dea21-df40-4698-90e5-7e16f65d77cc admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:31.814895 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702dea21-df40-4698-90e5-7e16f65d77cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:31.815194 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702dea21-df40-4698-90e5-7e16f65d77cc admin admin] PATH_INFO: `/v3/projects/07bc0c697e63498799ecf2a0335eae46/users/b1ca7ff32b434abc8d036ddfa266a584/roles/e1c88eebe911432cab6a07fbeb84653b` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:31.825084 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 787/1573] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jun 29 11:24:31 2022] POST /identity/v3/users => generated 287 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:24:31.825084 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985eb15a-e83a-44bc-89aa-44e6d4f9a239 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:24:31.830925 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702dea21-df40-4698-90e5-7e16f65d77cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=07bc0c697e63498799ecf2a0335eae46, user_id=b1ca7ff32b434abc8d036ddfa266a584, role_id=e1c88eebe911432cab6a07fbeb84653b)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:24:31.832370 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985eb15a-e83a-44bc-89aa-44e6d4f9a239 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:24:31.833737 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702dea21-df40-4698-90e5-7e16f65d77cc admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:24:31.838195 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0506b95a-d3fc-4126-a1f4-2ae25530391a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:24:31.838830 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0506b95a-d3fc-4126-a1f4-2ae25530391a admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:31.839122 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0506b95a-d3fc-4126-a1f4-2ae25530391a admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:31.839394 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0506b95a-d3fc-4126-a1f4-2ae25530391a admin admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:31.840127 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0506b95a-d3fc-4126-a1f4-2ae25530391a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:24:31.842032 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0506b95a-d3fc-4126-a1f4-2ae25530391a admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:24:31.890452 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 787/1574] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jun 29 11:24:31 2022] PUT /identity/v3/projects/07bc0c697e63498799ecf2a0335eae46/users/b1ca7ff32b434abc8d036ddfa266a584/roles/e1c88eebe911432cab6a07fbeb84653b => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:24:31.891316 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702dea21-df40-4698-90e5-7e16f65d77cc admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:24:31.918224 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 788/1575] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jun 29 11:24:31 2022] POST /identity/v3/projects => generated 389 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:24:31.919506 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0506b95a-d3fc-4126-a1f4-2ae25530391a admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:24:31.939215 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:24:31.960669 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702dea21-df40-4698-90e5-7e16f65d77cc admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:24:31.991605 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0506b95a-d3fc-4126-a1f4-2ae25530391a admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:24:31.997664 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49f9f63-9611-48ad-8a29-8ffd7f2cbf0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:24:31.998338 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49f9f63-9611-48ad-8a29-8ffd7f2cbf0e admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:31.998654 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49f9f63-9611-48ad-8a29-8ffd7f2cbf0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:31.998938 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49f9f63-9611-48ad-8a29-8ffd7f2cbf0e admin admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:32.000080 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49f9f63-9611-48ad-8a29-8ffd7f2cbf0e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:24:32.002456 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49f9f63-9611-48ad-8a29-8ffd7f2cbf0e admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:24:32.017980 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f8c867-efc8-4556-8aa1-cb44869971fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:24:32.018651 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f8c867-efc8-4556-8aa1-cb44869971fc admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:32.018931 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f8c867-efc8-4556-8aa1-cb44869971fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:32.019202 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f8c867-efc8-4556-8aa1-cb44869971fc admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:32.019908 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f8c867-efc8-4556-8aa1-cb44869971fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:24:32.021847 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f8c867-efc8-4556-8aa1-cb44869971fc admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:24:32.079247 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 789/1576] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jun 29 11:24:31 2022] POST /identity/v3/users => generated 289 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:24:32.080110 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49f9f63-9611-48ad-8a29-8ffd7f2cbf0e admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:24:32.090060 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49f9f63-9611-48ad-8a29-8ffd7f2cbf0e admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:24:32.094817 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569de7e6-8b6a-4d9a-bc1e-a14f14c2b09b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:24:32.095437 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569de7e6-8b6a-4d9a-bc1e-a14f14c2b09b admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:32.095720 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569de7e6-8b6a-4d9a-bc1e-a14f14c2b09b admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:32.096029 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569de7e6-8b6a-4d9a-bc1e-a14f14c2b09b admin admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:32.096710 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569de7e6-8b6a-4d9a-bc1e-a14f14c2b09b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:24:32.098694 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569de7e6-8b6a-4d9a-bc1e-a14f14c2b09b admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:24:32.104892 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 788/1577] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jun 29 11:24:31 2022] POST /identity/v3/users => generated 288 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:24:32.104892 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f8c867-efc8-4556-8aa1-cb44869971fc admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:24:32.112242 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f8c867-efc8-4556-8aa1-cb44869971fc admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:24:32.116878 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ac277d-e95e-4bf9-8cad-5eebf609db79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:24:32.117568 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ac277d-e95e-4bf9-8cad-5eebf609db79 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:32.117850 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ac277d-e95e-4bf9-8cad-5eebf609db79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:32.118129 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ac277d-e95e-4bf9-8cad-5eebf609db79 admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:32.118813 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ac277d-e95e-4bf9-8cad-5eebf609db79 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:24:32.120781 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ac277d-e95e-4bf9-8cad-5eebf609db79 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:24:32.200453 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 790/1578] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jun 29 11:24:32 2022] POST /identity/v3/projects => generated 383 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:24:32.201289 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569de7e6-8b6a-4d9a-bc1e-a14f14c2b09b admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:24:32.211317 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 789/1579] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jun 29 11:24:32 2022] POST /identity/v3/projects => generated 381 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:24:32.212166 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ac277d-e95e-4bf9-8cad-5eebf609db79 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:24:32.266467 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569de7e6-8b6a-4d9a-bc1e-a14f14c2b09b admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:24:32.283792 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ac277d-e95e-4bf9-8cad-5eebf609db79 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:24:32.288599 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4933d14c-b3b4-4935-9660-0740f831fe06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:24:32.289272 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4933d14c-b3b4-4935-9660-0740f831fe06 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:32.289559 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4933d14c-b3b4-4935-9660-0740f831fe06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:32.289849 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4933d14c-b3b4-4935-9660-0740f831fe06 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:32.290627 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4933d14c-b3b4-4935-9660-0740f831fe06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:24:32.292397 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4933d14c-b3b4-4935-9660-0740f831fe06 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:24:32.305972 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 790/1580] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:24:32 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 11:24:32.305972 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aac04c-00c8-48fd-b6da-775bf0c2e68b None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:32.305972 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aac04c-00c8-48fd-b6da-775bf0c2e68b None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:32.305972 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aac04c-00c8-48fd-b6da-775bf0c2e68b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:32.323105 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-d7aac04c-00c8-48fd-b6da-775bf0c2e68b None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:24:32.323771 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:24:32.325792 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-d7aac04c-00c8-48fd-b6da-775bf0c2e68b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:24:32.337251 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beaaf821-deed-4fb8-a18e-cae1722550dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:24:32.338268 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaaf821-deed-4fb8-a18e-cae1722550dc admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:32.341728 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaaf821-deed-4fb8-a18e-cae1722550dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:32.348136 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 791/1581] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:24:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:24:32.348864 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaaf821-deed-4fb8-a18e-cae1722550dc admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:32.348961 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7aac04c-00c8-48fd-b6da-775bf0c2e68b None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:24:32.349089 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beaaf821-deed-4fb8-a18e-cae1722550dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:24:32.349089 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beaaf821-deed-4fb8-a18e-cae1722550dc admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:24:32.350398 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7aac04c-00c8-48fd-b6da-775bf0c2e68b None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:24:32.356792 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 791/1582] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:24:32 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 11:24:32.356792 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beaaf821-deed-4fb8-a18e-cae1722550dc admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:24:32.357187 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee34ee65-9829-4e37-8182-a3a2731435f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:24:32.357857 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee34ee65-9829-4e37-8182-a3a2731435f0 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:32.358143 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee34ee65-9829-4e37-8182-a3a2731435f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:32.358420 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee34ee65-9829-4e37-8182-a3a2731435f0 admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:32.359111 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee34ee65-9829-4e37-8182-a3a2731435f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:24:32.361056 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee34ee65-9829-4e37-8182-a3a2731435f0 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:24:32.366003 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beaaf821-deed-4fb8-a18e-cae1722550dc admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:24:32.371727 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a891d8fe-8742-4015-9579-7e13d9d3aecf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:24:32.372421 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a891d8fe-8742-4015-9579-7e13d9d3aecf admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:32.372708 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a891d8fe-8742-4015-9579-7e13d9d3aecf admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:32.372984 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a891d8fe-8742-4015-9579-7e13d9d3aecf admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:32.373696 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a891d8fe-8742-4015-9579-7e13d9d3aecf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:24:32.375360 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a891d8fe-8742-4015-9579-7e13d9d3aecf admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:24:32.385796 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 792/1583] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:24:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 11:24:32.386609 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a891d8fe-8742-4015-9579-7e13d9d3aecf admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:24:32.395914 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a891d8fe-8742-4015-9579-7e13d9d3aecf admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:24:32.400468 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ebee85-f935-4f44-9679-15fe05b1fd0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:24:32.401093 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ebee85-f935-4f44-9679-15fe05b1fd0e admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:32.401381 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ebee85-f935-4f44-9679-15fe05b1fd0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:32.401665 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ebee85-f935-4f44-9679-15fe05b1fd0e admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:32.402406 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ebee85-f935-4f44-9679-15fe05b1fd0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:24:32.403933 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ebee85-f935-4f44-9679-15fe05b1fd0e admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:24:32.414890 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 793/1584] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:24:32 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 11:24:32.416985 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ebee85-f935-4f44-9679-15fe05b1fd0e admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:24:32.424780 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ebee85-f935-4f44-9679-15fe05b1fd0e admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:24:32.429179 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5719f80b-2bd1-40b9-9e6b-5be558bc5a88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:24:32.429783 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5719f80b-2bd1-40b9-9e6b-5be558bc5a88 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:32.430576 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5719f80b-2bd1-40b9-9e6b-5be558bc5a88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:32.430852 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5719f80b-2bd1-40b9-9e6b-5be558bc5a88 admin admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:32.431509 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5719f80b-2bd1-40b9-9e6b-5be558bc5a88 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:24:32.433401 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5719f80b-2bd1-40b9-9e6b-5be558bc5a88 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:24:32.439559 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 792/1585] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jun 29 11:24:32 2022] POST /identity/v3/users => generated 295 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:24:32.440466 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee34ee65-9829-4e37-8182-a3a2731435f0 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:24:32.452361 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee34ee65-9829-4e37-8182-a3a2731435f0 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:24:32.456899 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6bf11a-c00d-4357-9f32-caa6e4f57569 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:24:32.457542 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6bf11a-c00d-4357-9f32-caa6e4f57569 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:32.457942 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6bf11a-c00d-4357-9f32-caa6e4f57569 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:32.458152 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6bf11a-c00d-4357-9f32-caa6e4f57569 admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:32.458829 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6bf11a-c00d-4357-9f32-caa6e4f57569 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:24:32.460797 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6bf11a-c00d-4357-9f32-caa6e4f57569 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:24:32.526277 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 794/1586] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jun 29 11:24:32 2022] POST /identity/v3/users => generated 292 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:24:32.527155 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5719f80b-2bd1-40b9-9e6b-5be558bc5a88 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:24:32.536516 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5719f80b-2bd1-40b9-9e6b-5be558bc5a88 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:24:32.541673 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de81ce72-d617-4f68-900a-ddad2a92f25a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:24:32.542295 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de81ce72-d617-4f68-900a-ddad2a92f25a admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:32.542581 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de81ce72-d617-4f68-900a-ddad2a92f25a admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:32.542862 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de81ce72-d617-4f68-900a-ddad2a92f25a admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:32.543571 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de81ce72-d617-4f68-900a-ddad2a92f25a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:24:32.545207 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de81ce72-d617-4f68-900a-ddad2a92f25a admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:24:32.554500 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 793/1587] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jun 29 11:24:32 2022] POST /identity/v3/users => generated 291 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:24:32.555320 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6bf11a-c00d-4357-9f32-caa6e4f57569 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:24:32.558725 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 795/1588] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:24:32 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 11:24:32.558725 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de81ce72-d617-4f68-900a-ddad2a92f25a admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:24:32.567783 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de81ce72-d617-4f68-900a-ddad2a92f25a admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:24:32.568846 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:24:32.572793 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f35d87-b8d5-42c1-844b-a0127c804d32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:24:32.581340 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f35d87-b8d5-42c1-844b-a0127c804d32 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:32.581340 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f35d87-b8d5-42c1-844b-a0127c804d32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:32.581340 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f35d87-b8d5-42c1-844b-a0127c804d32 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:32.581340 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f35d87-b8d5-42c1-844b-a0127c804d32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:24:32.581340 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f35d87-b8d5-42c1-844b-a0127c804d32 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:24:32.588071 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 796/1589] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:24:32 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 11:24:32.588071 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f35d87-b8d5-42c1-844b-a0127c804d32 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:24:32.601107 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f35d87-b8d5-42c1-844b-a0127c804d32 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:24:32.606359 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13b7ec3-8807-4f75-a493-4790714cdd9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:24:32.607002 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13b7ec3-8807-4f75-a493-4790714cdd9f admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:32.610047 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13b7ec3-8807-4f75-a493-4790714cdd9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:32.610047 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13b7ec3-8807-4f75-a493-4790714cdd9f admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:32.610047 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13b7ec3-8807-4f75-a493-4790714cdd9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:24:32.610047 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13b7ec3-8807-4f75-a493-4790714cdd9f admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:24:32.613250 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6bf11a-c00d-4357-9f32-caa6e4f57569 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:24:32.619003 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fabde36-25f4-4bc7-aeb7-51972f3b77fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f4bb9f989e4024a5da28dced646b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:24:32.619675 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fabde36-25f4-4bc7-aeb7-51972f3b77fb service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:32.620007 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fabde36-25f4-4bc7-aeb7-51972f3b77fb service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:32.620287 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fabde36-25f4-4bc7-aeb7-51972f3b77fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:32.624412 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 797/1590] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:24:32 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 11:24:32.625240 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13b7ec3-8807-4f75-a493-4790714cdd9f admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:24:32.633893 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fabde36-25f4-4bc7-aeb7-51972f3b77fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:24:32.635174 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13b7ec3-8807-4f75-a493-4790714cdd9f admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:24:32.635355 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fabde36-25f4-4bc7-aeb7-51972f3b77fb service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:24:32.639853 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e60748f-bfa0-4585-8afc-314edfd28197 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:24:32.642649 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e60748f-bfa0-4585-8afc-314edfd28197 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:32.642649 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e60748f-bfa0-4585-8afc-314edfd28197 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:32.642649 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e60748f-bfa0-4585-8afc-314edfd28197 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:32.642649 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e60748f-bfa0-4585-8afc-314edfd28197 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:24:32.643527 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e60748f-bfa0-4585-8afc-314edfd28197 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:24:32.644978 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 794/1591] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:24:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:24:32.646398 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fabde36-25f4-4bc7-aeb7-51972f3b77fb service neutron] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:24:32.654325 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 798/1592] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:24:32 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 11:24:32.655162 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e60748f-bfa0-4585-8afc-314edfd28197 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:24:32.656874 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fabde36-25f4-4bc7-aeb7-51972f3b77fb service neutron] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:24:32.663284 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac5379d-be3b-42ac-9373-4b96d53a4ad5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:24:32.663991 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac5379d-be3b-42ac-9373-4b96d53a4ad5 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:32.664448 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac5379d-be3b-42ac-9373-4b96d53a4ad5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:32.664733 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac5379d-be3b-42ac-9373-4b96d53a4ad5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:32.665130 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e60748f-bfa0-4585-8afc-314edfd28197 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:24:32.665452 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac5379d-be3b-42ac-9373-4b96d53a4ad5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:24:32.667044 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac5379d-be3b-42ac-9373-4b96d53a4ad5 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:24:32.669693 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adc5d5c-6132-42c1-9a01-44d1bd8dc1c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:24:32.670345 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adc5d5c-6132-42c1-9a01-44d1bd8dc1c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:32.670627 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adc5d5c-6132-42c1-9a01-44d1bd8dc1c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:32.670903 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adc5d5c-6132-42c1-9a01-44d1bd8dc1c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:32.671625 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adc5d5c-6132-42c1-9a01-44d1bd8dc1c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:24:32.673268 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adc5d5c-6132-42c1-9a01-44d1bd8dc1c7 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:24:32.680893 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 795/1593] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:24:32 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 11:24:32.681850 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac5379d-be3b-42ac-9373-4b96d53a4ad5 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:24:32.686938 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 799/1594] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:24:32 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 11:24:32.686938 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adc5d5c-6132-42c1-9a01-44d1bd8dc1c7 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:24:32.693314 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac5379d-be3b-42ac-9373-4b96d53a4ad5 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:24:32.694299 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adc5d5c-6132-42c1-9a01-44d1bd8dc1c7 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:24:32.698476 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926e97cb-8be0-4502-9b5b-1786fe39edb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:24:32.699136 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926e97cb-8be0-4502-9b5b-1786fe39edb7 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:32.699263 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061526cf-b645-4869-a40b-028a6ac26a16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:24:32.699442 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926e97cb-8be0-4502-9b5b-1786fe39edb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:32.699729 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926e97cb-8be0-4502-9b5b-1786fe39edb7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:32.700139 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061526cf-b645-4869-a40b-028a6ac26a16 admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:32.700438 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061526cf-b645-4869-a40b-028a6ac26a16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:32.700559 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926e97cb-8be0-4502-9b5b-1786fe39edb7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:24:32.700704 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061526cf-b645-4869-a40b-028a6ac26a16 admin admin] PATH_INFO: `/v3/projects/e13ccc357c7c4b14b7f7a3ec6fdbfeb4/users/defbd94f64cd4861ab8aafb851206a8f/roles/e1c88eebe911432cab6a07fbeb84653b` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:32.702118 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926e97cb-8be0-4502-9b5b-1786fe39edb7 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:24:32.712993 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 796/1595] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:24:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 11:24:32.713718 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926e97cb-8be0-4502-9b5b-1786fe39edb7 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:24:32.717326 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061526cf-b645-4869-a40b-028a6ac26a16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e13ccc357c7c4b14b7f7a3ec6fdbfeb4, user_id=defbd94f64cd4861ab8aafb851206a8f, role_id=e1c88eebe911432cab6a07fbeb84653b)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:24:32.719077 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: WARNING py.warnings [None req-061526cf-b645-4869-a40b-028a6ac26a16 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 29 11:24:32.719077 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: warnings.warn(msg) Jun 29 11:24:32.719077 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: Jun 29 11:24:32.720861 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061526cf-b645-4869-a40b-028a6ac26a16 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:24:32.724665 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926e97cb-8be0-4502-9b5b-1786fe39edb7 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:24:32.729334 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb9fae0-fef4-441d-bc0e-7a5bf05b8978 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:24:32.730191 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb9fae0-fef4-441d-bc0e-7a5bf05b8978 admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:32.730470 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb9fae0-fef4-441d-bc0e-7a5bf05b8978 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:32.730743 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb9fae0-fef4-441d-bc0e-7a5bf05b8978 admin admin] PATH_INFO: `/v3/projects/bd0fad0a12a944dea5f22527241a6471/users/5a8f922dfd1f464f9b50a83521c610f1/roles/e1c88eebe911432cab6a07fbeb84653b` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:32.745877 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb9fae0-fef4-441d-bc0e-7a5bf05b8978 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd0fad0a12a944dea5f22527241a6471, user_id=5a8f922dfd1f464f9b50a83521c610f1, role_id=e1c88eebe911432cab6a07fbeb84653b)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:24:32.748547 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb9fae0-fef4-441d-bc0e-7a5bf05b8978 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:24:32.789508 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 800/1596] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jun 29 11:24:32 2022] PUT /identity/v3/projects/e13ccc357c7c4b14b7f7a3ec6fdbfeb4/users/defbd94f64cd4861ab8aafb851206a8f/roles/e1c88eebe911432cab6a07fbeb84653b => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:24:32.790393 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061526cf-b645-4869-a40b-028a6ac26a16 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:24:32.821744 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 797/1597] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jun 29 11:24:32 2022] PUT /identity/v3/projects/bd0fad0a12a944dea5f22527241a6471/users/5a8f922dfd1f464f9b50a83521c610f1/roles/e1c88eebe911432cab6a07fbeb84653b => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:24:32.821744 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb9fae0-fef4-441d-bc0e-7a5bf05b8978 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:24:32.862086 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061526cf-b645-4869-a40b-028a6ac26a16 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:24:32.888151 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb9fae0-fef4-441d-bc0e-7a5bf05b8978 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:24:32.891585 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:24:32.892988 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b8d4a6-65c8-411f-9bd6-d9fb806f7b04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:24:32.893975 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b8d4a6-65c8-411f-9bd6-d9fb806f7b04 admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:32.894477 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b8d4a6-65c8-411f-9bd6-d9fb806f7b04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:32.894669 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b8d4a6-65c8-411f-9bd6-d9fb806f7b04 admin admin] PATH_INFO: `/v3/projects/6815b6ed40654b9cb7ab3567b3f752a1/users/61eadda6693f47b680fb2083bf8bd573/roles/e1c88eebe911432cab6a07fbeb84653b` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:32.910480 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b8d4a6-65c8-411f-9bd6-d9fb806f7b04 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6815b6ed40654b9cb7ab3567b3f752a1, user_id=61eadda6693f47b680fb2083bf8bd573, role_id=e1c88eebe911432cab6a07fbeb84653b)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:24:32.913248 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b8d4a6-65c8-411f-9bd6-d9fb806f7b04 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:24:32.920761 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a3b3d9-f089-4255-b514-36a95783dc37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:24:32.924567 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a3b3d9-f089-4255-b514-36a95783dc37 admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:32.924567 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a3b3d9-f089-4255-b514-36a95783dc37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:32.924567 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a3b3d9-f089-4255-b514-36a95783dc37 admin admin] PATH_INFO: `/v3/projects/2c08a3d7e74f45999d1ba0ab11781458/users/f75e0b3528b14856b73dc441c52d1d73/roles/e1c88eebe911432cab6a07fbeb84653b` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:32.938558 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a3b3d9-f089-4255-b514-36a95783dc37 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c08a3d7e74f45999d1ba0ab11781458, user_id=f75e0b3528b14856b73dc441c52d1d73, role_id=e1c88eebe911432cab6a07fbeb84653b)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:24:32.941337 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a3b3d9-f089-4255-b514-36a95783dc37 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:24:32.994279 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 798/1598] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jun 29 11:24:32 2022] PUT /identity/v3/projects/6815b6ed40654b9cb7ab3567b3f752a1/users/61eadda6693f47b680fb2083bf8bd573/roles/e1c88eebe911432cab6a07fbeb84653b => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:24:32.994279 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b8d4a6-65c8-411f-9bd6-d9fb806f7b04 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:24:33.047673 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 801/1599] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jun 29 11:24:32 2022] PUT /identity/v3/projects/2c08a3d7e74f45999d1ba0ab11781458/users/f75e0b3528b14856b73dc441c52d1d73/roles/e1c88eebe911432cab6a07fbeb84653b => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:24:33.052707 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a3b3d9-f089-4255-b514-36a95783dc37 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:24:33.053120 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:24:33.064964 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b8d4a6-65c8-411f-9bd6-d9fb806f7b04 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:24:33.121241 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a3b3d9-f089-4255-b514-36a95783dc37 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:24:33.125533 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d394ec88-ba4b-455e-9296-b2907b2afaed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:24:33.125888 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42598305-6562-415b-beb8-420163e72456 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:24:33.126212 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d394ec88-ba4b-455e-9296-b2907b2afaed admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:33.126554 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42598305-6562-415b-beb8-420163e72456 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:33.126719 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d394ec88-ba4b-455e-9296-b2907b2afaed admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:33.126832 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42598305-6562-415b-beb8-420163e72456 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:33.127049 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d394ec88-ba4b-455e-9296-b2907b2afaed admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:33.127158 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42598305-6562-415b-beb8-420163e72456 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:33.127842 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42598305-6562-415b-beb8-420163e72456 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:24:33.128271 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d394ec88-ba4b-455e-9296-b2907b2afaed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:24:33.129466 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42598305-6562-415b-beb8-420163e72456 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:24:33.129899 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d394ec88-ba4b-455e-9296-b2907b2afaed admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:24:33.139859 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 802/1600] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:24:33 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 11:24:33.140680 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42598305-6562-415b-beb8-420163e72456 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:24:33.146415 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 799/1601] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:24:32 2022] GET /identity/v3/roles => generated 1420 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 11:24:33.146415 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5996100f-67fe-4ad0-a397-73ca7dbbe94e None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:33.146415 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5996100f-67fe-4ad0-a397-73ca7dbbe94e None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:33.146415 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5996100f-67fe-4ad0-a397-73ca7dbbe94e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:33.152536 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42598305-6562-415b-beb8-420163e72456 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:24:33.157348 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b44365-3f56-489d-9372-5511c9f0ebb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:24:33.158011 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b44365-3f56-489d-9372-5511c9f0ebb4 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:33.158292 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b44365-3f56-489d-9372-5511c9f0ebb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:33.158879 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b44365-3f56-489d-9372-5511c9f0ebb4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:33.159300 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b44365-3f56-489d-9372-5511c9f0ebb4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:24:33.160939 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b44365-3f56-489d-9372-5511c9f0ebb4 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:24:33.175277 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 803/1602] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:24:33 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 11:24:33.175277 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7de8ee-e0f3-467d-858a-937a5dd7a54f None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:33.175277 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7de8ee-e0f3-467d-858a-937a5dd7a54f None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:33.175277 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7de8ee-e0f3-467d-858a-937a5dd7a54f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:33.179555 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-5996100f-67fe-4ad0-a397-73ca7dbbe94e None None] MFA Rules not processed for user `defbd94f64cd4861ab8aafb851206a8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:24:33.180831 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-5996100f-67fe-4ad0-a397-73ca7dbbe94e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:24:33.203649 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-1d7de8ee-e0f3-467d-858a-937a5dd7a54f None None] MFA Rules not processed for user `5a8f922dfd1f464f9b50a83521c610f1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:24:33.205061 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-1d7de8ee-e0f3-467d-858a-937a5dd7a54f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:24:33.253583 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 800/1603] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:24:33 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:24:33.254743 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91630356-ec92-44e2-8aab-2b62a5625380 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:33.255107 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91630356-ec92-44e2-8aab-2b62a5625380 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:33.255460 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91630356-ec92-44e2-8aab-2b62a5625380 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:33.285929 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 804/1604] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:24:33 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:24:33.287213 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229528b7-2c75-48de-bd3f-a093ca163ba4 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:33.287613 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229528b7-2c75-48de-bd3f-a093ca163ba4 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:33.288021 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229528b7-2c75-48de-bd3f-a093ca163ba4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:33.321514 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-229528b7-2c75-48de-bd3f-a093ca163ba4 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:24:33.325328 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-229528b7-2c75-48de-bd3f-a093ca163ba4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:24:33.339182 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 805/1605] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:24:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:24:33.340340 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229528b7-2c75-48de-bd3f-a093ca163ba4 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:24:33.353063 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229528b7-2c75-48de-bd3f-a093ca163ba4 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:24:33.358042 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa067a03-d6bf-4dad-a677-7960bcdc9f30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:24:33.359313 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa067a03-d6bf-4dad-a677-7960bcdc9f30 admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:33.359608 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa067a03-d6bf-4dad-a677-7960bcdc9f30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:33.359922 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa067a03-d6bf-4dad-a677-7960bcdc9f30 admin admin] PATH_INFO: `/v3/projects/db854603a9334b68a6a482ba2bcfc0b9/users/75e79f6440b84ab2a9de551f2ff5b224/roles/e1c88eebe911432cab6a07fbeb84653b` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:33.378152 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa067a03-d6bf-4dad-a677-7960bcdc9f30 admin admin] RBAC: Authorizing `identity:create_grant(project_id=db854603a9334b68a6a482ba2bcfc0b9, user_id=75e79f6440b84ab2a9de551f2ff5b224, role_id=e1c88eebe911432cab6a07fbeb84653b)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:24:33.381139 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa067a03-d6bf-4dad-a677-7960bcdc9f30 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:24:33.428184 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-91630356-ec92-44e2-8aab-2b62a5625380 None None] MFA Rules not processed for user `61eadda6693f47b680fb2083bf8bd573`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:24:33.429419 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-91630356-ec92-44e2-8aab-2b62a5625380 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:24:33.478172 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:24:33.483307 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 806/1606] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jun 29 11:24:33 2022] PUT /identity/v3/projects/db854603a9334b68a6a482ba2bcfc0b9/users/75e79f6440b84ab2a9de551f2ff5b224/roles/e1c88eebe911432cab6a07fbeb84653b => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:24:33.484152 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa067a03-d6bf-4dad-a677-7960bcdc9f30 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:24:33.503178 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 801/1607] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:24:33 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:24:33.504241 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91630356-ec92-44e2-8aab-2b62a5625380 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:24:33.516815 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91630356-ec92-44e2-8aab-2b62a5625380 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:24:33.526854 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:24:33.558128 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa067a03-d6bf-4dad-a677-7960bcdc9f30 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:24:33.562784 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586858f9-1f57-4ab7-9466-3f9012534fc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:24:33.563682 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586858f9-1f57-4ab7-9466-3f9012534fc5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:33.563999 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586858f9-1f57-4ab7-9466-3f9012534fc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:33.564430 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586858f9-1f57-4ab7-9466-3f9012534fc5 admin admin] PATH_INFO: `/v3/projects/cbfba915f70342afb8267e34abc24bfe/users/8effcd6f6be64f14b3c8bc71f8866a99/roles/e1c88eebe911432cab6a07fbeb84653b` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:33.580278 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586858f9-1f57-4ab7-9466-3f9012534fc5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cbfba915f70342afb8267e34abc24bfe, user_id=8effcd6f6be64f14b3c8bc71f8866a99, role_id=e1c88eebe911432cab6a07fbeb84653b)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:24:33.583028 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586858f9-1f57-4ab7-9466-3f9012534fc5 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:24:33.585959 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b506cc17-aadb-473a-a531-3be4aa717807 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:24:33.587114 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b506cc17-aadb-473a-a531-3be4aa717807 admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:33.587486 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b506cc17-aadb-473a-a531-3be4aa717807 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:33.587762 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b506cc17-aadb-473a-a531-3be4aa717807 admin admin] PATH_INFO: `/v3/projects/a0273d33a17f4d74a1ec01c46107a6e4/users/5aff96e0fb2c4df58217ea77b5a6fa4c/roles/e1c88eebe911432cab6a07fbeb84653b` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:33.602738 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b506cc17-aadb-473a-a531-3be4aa717807 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0273d33a17f4d74a1ec01c46107a6e4, user_id=5aff96e0fb2c4df58217ea77b5a6fa4c, role_id=e1c88eebe911432cab6a07fbeb84653b)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:24:33.605587 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b506cc17-aadb-473a-a531-3be4aa717807 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:24:33.686620 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 807/1608] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jun 29 11:24:33 2022] PUT /identity/v3/projects/cbfba915f70342afb8267e34abc24bfe/users/8effcd6f6be64f14b3c8bc71f8866a99/roles/e1c88eebe911432cab6a07fbeb84653b => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:24:33.687632 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586858f9-1f57-4ab7-9466-3f9012534fc5 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:24:33.698211 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 802/1609] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jun 29 11:24:33 2022] PUT /identity/v3/projects/a0273d33a17f4d74a1ec01c46107a6e4/users/5aff96e0fb2c4df58217ea77b5a6fa4c/roles/e1c88eebe911432cab6a07fbeb84653b => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:24:33.699519 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3ffa12-d99a-461f-8455-4a76b52b1197 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:33.699806 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3ffa12-d99a-461f-8455-4a76b52b1197 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:33.700113 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3ffa12-d99a-461f-8455-4a76b52b1197 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:33.701541 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 803/1610] 10.222.0.44 () {54 vars in 961 bytes} [Wed Jun 29 11:24:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:24:33.702055 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3ffa12-d99a-461f-8455-4a76b52b1197 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:24:33.752148 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586858f9-1f57-4ab7-9466-3f9012534fc5 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:24:33.758056 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3ffa12-d99a-461f-8455-4a76b52b1197 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:24:33.761815 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfaf0e52-dd4b-4b87-a055-5267899f2318 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f4bb9f989e4024a5da28dced646b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:24:33.762461 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaf0e52-dd4b-4b87-a055-5267899f2318 service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:33.762745 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaf0e52-dd4b-4b87-a055-5267899f2318 service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:33.763020 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaf0e52-dd4b-4b87-a055-5267899f2318 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:33.814283 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80087c79-6b66-4244-9363-ad5411742a6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:24:33.814954 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80087c79-6b66-4244-9363-ad5411742a6b admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:33.815238 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80087c79-6b66-4244-9363-ad5411742a6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:33.816273 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80087c79-6b66-4244-9363-ad5411742a6b admin admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:33.816498 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80087c79-6b66-4244-9363-ad5411742a6b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:24:33.818761 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80087c79-6b66-4244-9363-ad5411742a6b admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:24:33.838962 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfaf0e52-dd4b-4b87-a055-5267899f2318 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:24:33.840497 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfaf0e52-dd4b-4b87-a055-5267899f2318 service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:24:33.850643 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 804/1611] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:24:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:24:33.852621 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34fa287-4d78-4530-9096-07dc531110fc None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:33.852910 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34fa287-4d78-4530-9096-07dc531110fc None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:33.853207 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34fa287-4d78-4530-9096-07dc531110fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:33.879793 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-e34fa287-4d78-4530-9096-07dc531110fc None None] MFA Rules not processed for user `75e79f6440b84ab2a9de551f2ff5b224`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:24:33.881175 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-e34fa287-4d78-4530-9096-07dc531110fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:24:33.915854 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:24:33.945780 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 805/1612] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:24:33 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:24:33.947109 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61830c4d-3788-48d5-bbae-1de19b8a42c4 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:33.947400 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61830c4d-3788-48d5-bbae-1de19b8a42c4 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:33.947680 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61830c4d-3788-48d5-bbae-1de19b8a42c4 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:33.949126 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 806/1613] 10.222.0.44 () {54 vars in 961 bytes} [Wed Jun 29 11:24:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:24:33.950006 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186059fe-61bb-42ec-8058-ec461b821d95 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:33.950289 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186059fe-61bb-42ec-8058-ec461b821d95 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:33.950577 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186059fe-61bb-42ec-8058-ec461b821d95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:33.964584 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 808/1614] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jun 29 11:24:33 2022] POST /identity/v3/projects => generated 371 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:24:33.964584 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e590a9-684a-4768-8924-ad4d28cd5b24 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:33.964584 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e590a9-684a-4768-8924-ad4d28cd5b24 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:33.964584 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e590a9-684a-4768-8924-ad4d28cd5b24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:33.970425 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-186059fe-61bb-42ec-8058-ec461b821d95 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:24:33.971683 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-186059fe-61bb-42ec-8058-ec461b821d95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:24:34.002533 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-62e590a9-684a-4768-8924-ad4d28cd5b24 None None] MFA Rules not processed for user `5aff96e0fb2c4df58217ea77b5a6fa4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:24:34.004025 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-62e590a9-684a-4768-8924-ad4d28cd5b24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:24:34.043041 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 807/1615] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:24:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:24:34.045699 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82345af9-7e81-4f9a-994d-5312a5dd64a8 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:34.046249 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82345af9-7e81-4f9a-994d-5312a5dd64a8 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:34.046828 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82345af9-7e81-4f9a-994d-5312a5dd64a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:34.077026 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-82345af9-7e81-4f9a-994d-5312a5dd64a8 None None] MFA Rules not processed for user `796aeeff16df4739bddec1f6dee61bfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:24:34.078815 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-82345af9-7e81-4f9a-994d-5312a5dd64a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:24:34.108472 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:24:34.113225 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 809/1616] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:24:33 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:24:34.114341 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e590a9-684a-4768-8924-ad4d28cd5b24 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:24:34.125397 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e590a9-684a-4768-8924-ad4d28cd5b24 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:24:34.134440 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713eb384-d214-4406-96b5-a39c452a169b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:24:34.135668 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713eb384-d214-4406-96b5-a39c452a169b admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:34.136303 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713eb384-d214-4406-96b5-a39c452a169b admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:34.137219 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713eb384-d214-4406-96b5-a39c452a169b admin admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:34.138599 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713eb384-d214-4406-96b5-a39c452a169b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:24:34.141461 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713eb384-d214-4406-96b5-a39c452a169b admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:24:34.155029 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 808/1617] 10.222.0.44 () {58 vars in 1058 bytes} [Wed Jun 29 11:24:34 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:24:34.156762 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c9d8f8-6ebf-4028-87c4-802d24547c92 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:34.157194 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c9d8f8-6ebf-4028-87c4-802d24547c92 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:34.157666 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c9d8f8-6ebf-4028-87c4-802d24547c92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:34.177437 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-39c9d8f8-6ebf-4028-87c4-802d24547c92 None None] MFA Rules not processed for user `796aeeff16df4739bddec1f6dee61bfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:24:34.178757 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-39c9d8f8-6ebf-4028-87c4-802d24547c92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:24:34.189252 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 809/1618] 10.222.0.44 () {58 vars in 1058 bytes} [Wed Jun 29 11:24:34 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:24:34.190144 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c9d8f8-6ebf-4028-87c4-802d24547c92 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:24:34.202553 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c9d8f8-6ebf-4028-87c4-802d24547c92 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:24:34.208769 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4ac955-e26a-4ff6-bea2-037d84e4b7c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:24:34.209614 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4ac955-e26a-4ff6-bea2-037d84e4b7c8 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:34.209984 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4ac955-e26a-4ff6-bea2-037d84e4b7c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:34.210325 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4ac955-e26a-4ff6-bea2-037d84e4b7c8 admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:34.211093 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4ac955-e26a-4ff6-bea2-037d84e4b7c8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:24:34.213677 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4ac955-e26a-4ff6-bea2-037d84e4b7c8 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:24:34.235971 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 810/1619] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jun 29 11:24:34 2022] POST /identity/v3/projects => generated 387 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:24:34.235971 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713eb384-d214-4406-96b5-a39c452a169b admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:24:34.237517 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-713eb384-d214-4406-96b5-a39c452a169b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:24:34.309865 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 810/1620] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jun 29 11:24:34 2022] POST /identity/v3/users => generated 292 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:24:34.309865 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4ac955-e26a-4ff6-bea2-037d84e4b7c8 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:24:34.310881 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713eb384-d214-4406-96b5-a39c452a169b admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:24:34.315277 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ee931a-fe91-45ac-8069-eaff7ff9d4a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f4bb9f989e4024a5da28dced646b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:24:34.316180 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ee931a-fe91-45ac-8069-eaff7ff9d4a3 service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:34.316618 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ee931a-fe91-45ac-8069-eaff7ff9d4a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:34.316901 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ee931a-fe91-45ac-8069-eaff7ff9d4a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:34.326542 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4ac955-e26a-4ff6-bea2-037d84e4b7c8 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:24:34.396301 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91fc84d-07d9-43fc-91d7-5dbff836bd39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:24:34.397061 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91fc84d-07d9-43fc-91d7-5dbff836bd39 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:34.397348 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91fc84d-07d9-43fc-91d7-5dbff836bd39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:34.397624 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91fc84d-07d9-43fc-91d7-5dbff836bd39 admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:34.404081 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ee931a-fe91-45ac-8069-eaff7ff9d4a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:24:34.406001 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91fc84d-07d9-43fc-91d7-5dbff836bd39 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:24:34.406703 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ee931a-fe91-45ac-8069-eaff7ff9d4a3 service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:24:34.408055 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91fc84d-07d9-43fc-91d7-5dbff836bd39 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:24:34.425427 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 811/1621] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:24:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:24:34.425427 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ee931a-fe91-45ac-8069-eaff7ff9d4a3 service neutron] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:24:34.501696 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ee931a-fe91-45ac-8069-eaff7ff9d4a3 service neutron] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:24:34.509630 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02925b7f-108c-4f80-b360-679e02e9e4e7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '796aeeff16df4739bddec1f6dee61bfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:24:34.510537 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02925b7f-108c-4f80-b360-679e02e9e4e7 service nova] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:34.511034 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02925b7f-108c-4f80-b360-679e02e9e4e7 service nova] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:34.511542 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02925b7f-108c-4f80-b360-679e02e9e4e7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:34.540526 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 811/1622] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jun 29 11:24:34 2022] POST /identity/v3/projects => generated 381 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:24:34.542024 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91fc84d-07d9-43fc-91d7-5dbff836bd39 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:24:34.582505 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:24:34.636835 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02925b7f-108c-4f80-b360-679e02e9e4e7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:24:34.639064 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02925b7f-108c-4f80-b360-679e02e9e4e7 service nova] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:24:34.652313 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 812/1623] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:24:34 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:24:34.663652 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91fc84d-07d9-43fc-91d7-5dbff836bd39 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:24:34.664914 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02925b7f-108c-4f80-b360-679e02e9e4e7 service nova] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:24:34.672563 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616fb947-8832-429b-a87e-2a4a606fd356 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '796aeeff16df4739bddec1f6dee61bfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:24:34.673250 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616fb947-8832-429b-a87e-2a4a606fd356 service nova] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:34.673583 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616fb947-8832-429b-a87e-2a4a606fd356 service nova] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:34.673865 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616fb947-8832-429b-a87e-2a4a606fd356 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:34.679076 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02925b7f-108c-4f80-b360-679e02e9e4e7 service nova] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:24:34.737242 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:24:34.765082 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616fb947-8832-429b-a87e-2a4a606fd356 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:24:34.767197 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616fb947-8832-429b-a87e-2a4a606fd356 service nova] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:24:34.779929 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 812/1624] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:24:34 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:24:34.780854 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616fb947-8832-429b-a87e-2a4a606fd356 service nova] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:24:34.792444 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c06430-04c9-4d5e-a7f4-ca449eca9f6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:24:34.793394 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c06430-04c9-4d5e-a7f4-ca449eca9f6a admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:34.795494 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c06430-04c9-4d5e-a7f4-ca449eca9f6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:34.795965 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c06430-04c9-4d5e-a7f4-ca449eca9f6a admin admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:34.796844 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c06430-04c9-4d5e-a7f4-ca449eca9f6a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:24:34.801803 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c06430-04c9-4d5e-a7f4-ca449eca9f6a admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:24:34.907560 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616fb947-8832-429b-a87e-2a4a606fd356 service nova] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:24:34.920143 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107200d4-2dba-4698-b4eb-75d05f8f8fe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:24:34.922997 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107200d4-2dba-4698-b4eb-75d05f8f8fe0 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:34.926269 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107200d4-2dba-4698-b4eb-75d05f8f8fe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:34.926269 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107200d4-2dba-4698-b4eb-75d05f8f8fe0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:34.926269 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107200d4-2dba-4698-b4eb-75d05f8f8fe0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:24:34.928907 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107200d4-2dba-4698-b4eb-75d05f8f8fe0 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:24:34.953498 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 813/1625] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jun 29 11:24:34 2022] POST /identity/v3/users => generated 300 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:24:34.953498 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c06430-04c9-4d5e-a7f4-ca449eca9f6a admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:24:34.987880 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 813/1626] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:24:34 2022] GET /identity/v3/roles => generated 1420 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 11:24:34.990039 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107200d4-2dba-4698-b4eb-75d05f8f8fe0 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:24:34.999177 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c06430-04c9-4d5e-a7f4-ca449eca9f6a admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:24:35.013616 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d4131a-732d-4ed1-9acb-9a2bb73d1728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:24:35.014585 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d4131a-732d-4ed1-9acb-9a2bb73d1728 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:35.016057 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d4131a-732d-4ed1-9acb-9a2bb73d1728 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:35.017294 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d4131a-732d-4ed1-9acb-9a2bb73d1728 admin admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:35.021131 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d4131a-732d-4ed1-9acb-9a2bb73d1728 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:24:35.023634 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d4131a-732d-4ed1-9acb-9a2bb73d1728 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:24:35.026274 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107200d4-2dba-4698-b4eb-75d05f8f8fe0 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:24:35.040454 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddf8102-c86c-47a6-8916-bf97c60ebe20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:24:35.041302 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddf8102-c86c-47a6-8916-bf97c60ebe20 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:35.041591 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddf8102-c86c-47a6-8916-bf97c60ebe20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:35.041866 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddf8102-c86c-47a6-8916-bf97c60ebe20 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:35.042599 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddf8102-c86c-47a6-8916-bf97c60ebe20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:24:35.047789 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddf8102-c86c-47a6-8916-bf97c60ebe20 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:24:35.072457 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 814/1627] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:24:34 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 11:24:35.075129 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddf8102-c86c-47a6-8916-bf97c60ebe20 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:24:35.092774 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddf8102-c86c-47a6-8916-bf97c60ebe20 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:24:35.100379 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8924ea4-e367-4cd1-9e31-473b25c96658 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:24:35.101789 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8924ea4-e367-4cd1-9e31-473b25c96658 admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:35.102373 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8924ea4-e367-4cd1-9e31-473b25c96658 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:35.102825 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8924ea4-e367-4cd1-9e31-473b25c96658 admin admin] PATH_INFO: `/v3/projects/3430f0551c3c45ceaf2436a191d0fb45/users/4a8af2f7029e45a69b0fbcb986bbeafd/roles/e95c2ce63e114cc19a9dda6ad21e794f` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:35.119025 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8924ea4-e367-4cd1-9e31-473b25c96658 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3430f0551c3c45ceaf2436a191d0fb45, user_id=4a8af2f7029e45a69b0fbcb986bbeafd, role_id=e95c2ce63e114cc19a9dda6ad21e794f)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:24:35.121755 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8924ea4-e367-4cd1-9e31-473b25c96658 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:24:35.182929 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 814/1628] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jun 29 11:24:34 2022] POST /identity/v3/users => generated 297 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:24:35.184908 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d4131a-732d-4ed1-9acb-9a2bb73d1728 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:24:35.206537 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d4131a-732d-4ed1-9acb-9a2bb73d1728 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:24:35.228662 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ec2f69-1fd4-44d4-9d3a-ca868204ba73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:24:35.228662 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ec2f69-1fd4-44d4-9d3a-ca868204ba73 admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:35.228662 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ec2f69-1fd4-44d4-9d3a-ca868204ba73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:35.228662 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ec2f69-1fd4-44d4-9d3a-ca868204ba73 admin admin] PATH_INFO: `/v3/projects/d8508b04f40947389176c04e55368fb4/users/7851a39b35a442f0b062de78adf7fead/roles/e95c2ce63e114cc19a9dda6ad21e794f` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:35.236730 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:24:35.247986 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 815/1629] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jun 29 11:24:35 2022] PUT /identity/v3/projects/3430f0551c3c45ceaf2436a191d0fb45/users/4a8af2f7029e45a69b0fbcb986bbeafd/roles/e95c2ce63e114cc19a9dda6ad21e794f => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:24:35.249225 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216a4ebb-82d4-424e-ab3b-77ed0d4c8ad1 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:35.249617 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216a4ebb-82d4-424e-ab3b-77ed0d4c8ad1 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:35.250949 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216a4ebb-82d4-424e-ab3b-77ed0d4c8ad1 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:35.253445 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 816/1630] 10.222.0.44 () {54 vars in 912 bytes} [Wed Jun 29 11:24:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:24:35.256776 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216a4ebb-82d4-424e-ab3b-77ed0d4c8ad1 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:24:35.278075 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ec2f69-1fd4-44d4-9d3a-ca868204ba73 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8508b04f40947389176c04e55368fb4, user_id=7851a39b35a442f0b062de78adf7fead, role_id=e95c2ce63e114cc19a9dda6ad21e794f)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:24:35.280843 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ec2f69-1fd4-44d4-9d3a-ca868204ba73 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:24:35.363885 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 815/1631] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jun 29 11:24:35 2022] PUT /identity/v3/projects/d8508b04f40947389176c04e55368fb4/users/7851a39b35a442f0b062de78adf7fead/roles/e95c2ce63e114cc19a9dda6ad21e794f => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:24:35.363885 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ec2f69-1fd4-44d4-9d3a-ca868204ba73 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:24:35.366632 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216a4ebb-82d4-424e-ab3b-77ed0d4c8ad1 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:24:35.394739 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:24:35.433461 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abebcca6-796c-4a5d-b1a8-f1312961aa6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:24:35.433841 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abebcca6-796c-4a5d-b1a8-f1312961aa6d admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:35.434308 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abebcca6-796c-4a5d-b1a8-f1312961aa6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:35.434656 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abebcca6-796c-4a5d-b1a8-f1312961aa6d admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:35.435454 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abebcca6-796c-4a5d-b1a8-f1312961aa6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:24:35.437302 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abebcca6-796c-4a5d-b1a8-f1312961aa6d admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:24:35.443167 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ec2f69-1fd4-44d4-9d3a-ca868204ba73 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:24:35.452338 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 817/1632] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:24:35 2022] GET /identity/v3/roles => generated 1420 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 11:24:35.453670 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ca8fcd-f08d-4091-b792-24aa37e74050 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:24:35.456513 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dfdca3-f497-4d50-8005-1072e8b42303 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:35.456513 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dfdca3-f497-4d50-8005-1072e8b42303 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:35.456513 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dfdca3-f497-4d50-8005-1072e8b42303 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:35.456766 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ca8fcd-f08d-4091-b792-24aa37e74050 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:35.456766 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ca8fcd-f08d-4091-b792-24aa37e74050 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:35.456766 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ca8fcd-f08d-4091-b792-24aa37e74050 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:35.457912 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ca8fcd-f08d-4091-b792-24aa37e74050 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:24:35.461400 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ca8fcd-f08d-4091-b792-24aa37e74050 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:24:35.476265 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 816/1633] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:24:35 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 11:24:35.478933 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204d5ce0-40ca-47d4-803d-1a166eed9e44 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:35.482791 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204d5ce0-40ca-47d4-803d-1a166eed9e44 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:35.482791 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204d5ce0-40ca-47d4-803d-1a166eed9e44 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:35.482982 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-77dfdca3-f497-4d50-8005-1072e8b42303 None None] MFA Rules not processed for user `796aeeff16df4739bddec1f6dee61bfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:24:35.483067 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 817/1634] 10.222.0.44 () {54 vars in 912 bytes} [Wed Jun 29 11:24:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:24:35.483149 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-77dfdca3-f497-4d50-8005-1072e8b42303 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:24:35.483224 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204d5ce0-40ca-47d4-803d-1a166eed9e44 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:24:35.500569 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204d5ce0-40ca-47d4-803d-1a166eed9e44 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:24:35.507299 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1822ba-8d49-4812-9604-a221f4bd4385 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:24:35.508788 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1822ba-8d49-4812-9604-a221f4bd4385 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:35.510396 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1822ba-8d49-4812-9604-a221f4bd4385 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:35.510396 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1822ba-8d49-4812-9604-a221f4bd4385 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:35.510396 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1822ba-8d49-4812-9604-a221f4bd4385 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:24:35.512983 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1822ba-8d49-4812-9604-a221f4bd4385 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:24:35.529644 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 818/1635] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:24:35 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 11:24:35.530837 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1822ba-8d49-4812-9604-a221f4bd4385 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:24:35.542532 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1822ba-8d49-4812-9604-a221f4bd4385 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:24:35.548739 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0295d203-5021-4fb0-a7bf-25df166d4e56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:24:35.549930 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0295d203-5021-4fb0-a7bf-25df166d4e56 admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:35.550307 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0295d203-5021-4fb0-a7bf-25df166d4e56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:35.550981 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0295d203-5021-4fb0-a7bf-25df166d4e56 admin admin] PATH_INFO: `/v3/projects/286480ac54ae4066bbf33c25f59c0cdb/users/f015bb491e3b42ca928d05e49ecea011/roles/e95c2ce63e114cc19a9dda6ad21e794f` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:35.572867 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0295d203-5021-4fb0-a7bf-25df166d4e56 admin admin] RBAC: Authorizing `identity:create_grant(project_id=286480ac54ae4066bbf33c25f59c0cdb, user_id=f015bb491e3b42ca928d05e49ecea011, role_id=e95c2ce63e114cc19a9dda6ad21e794f)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:24:35.575719 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0295d203-5021-4fb0-a7bf-25df166d4e56 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:24:35.619431 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 818/1636] 10.222.0.44 () {58 vars in 1009 bytes} [Wed Jun 29 11:24:35 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:24:35.620393 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dfdca3-f497-4d50-8005-1072e8b42303 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:24:35.632913 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dfdca3-f497-4d50-8005-1072e8b42303 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:24:35.641007 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1a2887-18d0-415c-9d9a-11fb5b3f6187 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:24:35.642010 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1a2887-18d0-415c-9d9a-11fb5b3f6187 admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:35.642198 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1a2887-18d0-415c-9d9a-11fb5b3f6187 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:35.642506 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1a2887-18d0-415c-9d9a-11fb5b3f6187 admin admin] PATH_INFO: `/v3/projects/3430f0551c3c45ceaf2436a191d0fb45/users/4a8af2f7029e45a69b0fbcb986bbeafd/roles/e95c2ce63e114cc19a9dda6ad21e794f` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:35.645089 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1a2887-18d0-415c-9d9a-11fb5b3f6187 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3430f0551c3c45ceaf2436a191d0fb45, user_id=4a8af2f7029e45a69b0fbcb986bbeafd, role_id=e95c2ce63e114cc19a9dda6ad21e794f)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:24:35.647363 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1a2887-18d0-415c-9d9a-11fb5b3f6187 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:24:35.661440 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 819/1637] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jun 29 11:24:35 2022] PUT /identity/v3/projects/3430f0551c3c45ceaf2436a191d0fb45/users/4a8af2f7029e45a69b0fbcb986bbeafd/roles/e95c2ce63e114cc19a9dda6ad21e794f => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:24:35.663797 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578d5c14-f5cb-433e-8c22-00a068dbe227 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:35.739088 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578d5c14-f5cb-433e-8c22-00a068dbe227 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:35.739088 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578d5c14-f5cb-433e-8c22-00a068dbe227 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:35.762645 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 819/1638] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jun 29 11:24:35 2022] PUT /identity/v3/projects/286480ac54ae4066bbf33c25f59c0cdb/users/f015bb491e3b42ca928d05e49ecea011/roles/e95c2ce63e114cc19a9dda6ad21e794f => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:24:35.762645 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0295d203-5021-4fb0-a7bf-25df166d4e56 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:24:35.763631 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-578d5c14-f5cb-433e-8c22-00a068dbe227 None None] MFA Rules not processed for user `796aeeff16df4739bddec1f6dee61bfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:24:35.763631 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-578d5c14-f5cb-433e-8c22-00a068dbe227 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:24:35.790963 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0295d203-5021-4fb0-a7bf-25df166d4e56 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:24:35.800684 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b19b71-d7cf-42a2-9a8e-4dd5954d970c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:24:35.801623 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b19b71-d7cf-42a2-9a8e-4dd5954d970c admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:35.801935 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b19b71-d7cf-42a2-9a8e-4dd5954d970c admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:35.802209 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b19b71-d7cf-42a2-9a8e-4dd5954d970c admin admin] PATH_INFO: `/v3/projects/d8508b04f40947389176c04e55368fb4/users/7851a39b35a442f0b062de78adf7fead/roles/e95c2ce63e114cc19a9dda6ad21e794f` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:35.805227 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b19b71-d7cf-42a2-9a8e-4dd5954d970c admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8508b04f40947389176c04e55368fb4, user_id=7851a39b35a442f0b062de78adf7fead, role_id=e95c2ce63e114cc19a9dda6ad21e794f)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:24:35.808290 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b19b71-d7cf-42a2-9a8e-4dd5954d970c admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:24:35.830024 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 820/1639] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jun 29 11:24:35 2022] PUT /identity/v3/projects/d8508b04f40947389176c04e55368fb4/users/7851a39b35a442f0b062de78adf7fead/roles/e95c2ce63e114cc19a9dda6ad21e794f => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:24:35.830024 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b19b71-d7cf-42a2-9a8e-4dd5954d970c admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:24:35.904499 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b19b71-d7cf-42a2-9a8e-4dd5954d970c admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:24:35.908924 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b9f3fc-da8f-4ef9-87d2-9bc8343b5cb3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff7931b75a924394b1af733febce0834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:24:35.910037 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b9f3fc-da8f-4ef9-87d2-9bc8343b5cb3 service cinder] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:35.910464 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b9f3fc-da8f-4ef9-87d2-9bc8343b5cb3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:35.910898 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b9f3fc-da8f-4ef9-87d2-9bc8343b5cb3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:35.918382 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 820/1640] 10.222.0.44 () {58 vars in 1009 bytes} [Wed Jun 29 11:24:35 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:24:35.920299 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5987951e-dc0b-4e20-b0d6-b8d3bf846180 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:35.920299 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5987951e-dc0b-4e20-b0d6-b8d3bf846180 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:35.920486 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5987951e-dc0b-4e20-b0d6-b8d3bf846180 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:35.974307 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:24:35.975521 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-5987951e-dc0b-4e20-b0d6-b8d3bf846180 None None] MFA Rules not processed for user `4a8af2f7029e45a69b0fbcb986bbeafd`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:24:35.977228 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-5987951e-dc0b-4e20-b0d6-b8d3bf846180 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:24:36.021753 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b9f3fc-da8f-4ef9-87d2-9bc8343b5cb3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:24:36.025091 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b9f3fc-da8f-4ef9-87d2-9bc8343b5cb3 service cinder] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:24:36.039614 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 821/1641] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:24:35 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:24:36.041193 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b9f3fc-da8f-4ef9-87d2-9bc8343b5cb3 service cinder] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:24:36.105383 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 821/1642] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:24:35 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:24:36.107450 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f961266-dcc8-4fd6-bb87-e1db0d6081e4 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:36.108975 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f961266-dcc8-4fd6-bb87-e1db0d6081e4 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:36.108975 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f961266-dcc8-4fd6-bb87-e1db0d6081e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:36.146548 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b9f3fc-da8f-4ef9-87d2-9bc8343b5cb3 service cinder] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:24:36.151683 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc69074-4fe4-45b4-9e40-ad989e7d7787 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:24:36.153076 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc69074-4fe4-45b4-9e40-ad989e7d7787 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:36.153361 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc69074-4fe4-45b4-9e40-ad989e7d7787 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:36.153797 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc69074-4fe4-45b4-9e40-ad989e7d7787 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:36.154676 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc69074-4fe4-45b4-9e40-ad989e7d7787 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:24:36.156560 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc69074-4fe4-45b4-9e40-ad989e7d7787 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:24:36.174980 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 822/1643] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:24:36 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 11:24:36.175904 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc69074-4fe4-45b4-9e40-ad989e7d7787 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:24:36.187694 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-3f961266-dcc8-4fd6-bb87-e1db0d6081e4 None None] MFA Rules not processed for user `7851a39b35a442f0b062de78adf7fead`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:24:36.189232 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-3f961266-dcc8-4fd6-bb87-e1db0d6081e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:24:36.197308 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc69074-4fe4-45b4-9e40-ad989e7d7787 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:24:36.205354 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e029943a-d658-4462-8e51-ad4ace9d2979 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff7931b75a924394b1af733febce0834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:24:36.208156 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e029943a-d658-4462-8e51-ad4ace9d2979 service cinder] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:36.208505 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e029943a-d658-4462-8e51-ad4ace9d2979 service cinder] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:36.208844 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e029943a-d658-4462-8e51-ad4ace9d2979 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:36.221687 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:24:36.233701 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e029943a-d658-4462-8e51-ad4ace9d2979 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:24:36.235161 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e029943a-d658-4462-8e51-ad4ace9d2979 service cinder] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:24:36.261617 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 823/1644] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:24:36 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:24:36.262481 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e029943a-d658-4462-8e51-ad4ace9d2979 service cinder] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:24:36.283324 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e029943a-d658-4462-8e51-ad4ace9d2979 service cinder] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:24:36.299182 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cd2e95-6ae0-4291-8af6-aaa34e74d4e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:24:36.300913 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cd2e95-6ae0-4291-8af6-aaa34e74d4e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:36.301798 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cd2e95-6ae0-4291-8af6-aaa34e74d4e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:36.301798 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cd2e95-6ae0-4291-8af6-aaa34e74d4e3 admin admin] PATH_INFO: `/v3/projects/286480ac54ae4066bbf33c25f59c0cdb/users/f015bb491e3b42ca928d05e49ecea011/roles/e95c2ce63e114cc19a9dda6ad21e794f` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:36.308690 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cd2e95-6ae0-4291-8af6-aaa34e74d4e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=286480ac54ae4066bbf33c25f59c0cdb, user_id=f015bb491e3b42ca928d05e49ecea011, role_id=e95c2ce63e114cc19a9dda6ad21e794f)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:24:36.319194 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cd2e95-6ae0-4291-8af6-aaa34e74d4e3 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:24:36.355613 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 824/1645] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jun 29 11:24:36 2022] PUT /identity/v3/projects/286480ac54ae4066bbf33c25f59c0cdb/users/f015bb491e3b42ca928d05e49ecea011/roles/e95c2ce63e114cc19a9dda6ad21e794f => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:24:36.358537 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c842c28-334d-444f-a23a-77fa4590edd9 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:36.359105 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c842c28-334d-444f-a23a-77fa4590edd9 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:36.359539 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c842c28-334d-444f-a23a-77fa4590edd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:36.367397 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 822/1646] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:24:36 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:24:36.413012 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f961266-dcc8-4fd6-bb87-e1db0d6081e4 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:24:36.433157 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-0c842c28-334d-444f-a23a-77fa4590edd9 None None] MFA Rules not processed for user `f015bb491e3b42ca928d05e49ecea011`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:24:36.433157 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-0c842c28-334d-444f-a23a-77fa4590edd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:24:36.442483 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f961266-dcc8-4fd6-bb87-e1db0d6081e4 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:24:36.623649 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682ac1e3-abf6-44fa-91f9-7fa673a0216b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff7931b75a924394b1af733febce0834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:24:36.624403 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682ac1e3-abf6-44fa-91f9-7fa673a0216b service cinder] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:36.624688 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682ac1e3-abf6-44fa-91f9-7fa673a0216b service cinder] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:36.624970 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682ac1e3-abf6-44fa-91f9-7fa673a0216b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:36.629436 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 825/1647] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:24:36 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:24:36.744844 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682ac1e3-abf6-44fa-91f9-7fa673a0216b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:24:36.746792 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682ac1e3-abf6-44fa-91f9-7fa673a0216b service cinder] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:24:36.761462 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 823/1648] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:24:36 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 11:24:36.910623 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c842c28-334d-444f-a23a-77fa4590edd9 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:24:36.938887 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c842c28-334d-444f-a23a-77fa4590edd9 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:24:36.952462 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7803f444-6418-4c67-8f4e-ee7667d1a802 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff7931b75a924394b1af733febce0834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:24:36.953131 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7803f444-6418-4c67-8f4e-ee7667d1a802 service cinder] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:36.953420 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7803f444-6418-4c67-8f4e-ee7667d1a802 service cinder] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:36.953698 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7803f444-6418-4c67-8f4e-ee7667d1a802 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:37.051050 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7803f444-6418-4c67-8f4e-ee7667d1a802 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:24:37.052725 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7803f444-6418-4c67-8f4e-ee7667d1a802 service cinder] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:24:37.091886 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 826/1649] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:24:36 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:24:37.273078 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:24:37.296561 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682ac1e3-abf6-44fa-91f9-7fa673a0216b service cinder] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:24:37.305908 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682ac1e3-abf6-44fa-91f9-7fa673a0216b service cinder] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:24:37.321144 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7b6f38-0bcd-4eba-9344-a5a1839a6ee6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff7931b75a924394b1af733febce0834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:24:37.325688 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7b6f38-0bcd-4eba-9344-a5a1839a6ee6 service cinder] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:37.328379 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7b6f38-0bcd-4eba-9344-a5a1839a6ee6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:37.328935 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7b6f38-0bcd-4eba-9344-a5a1839a6ee6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:37.359352 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:24:37.360880 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7b6f38-0bcd-4eba-9344-a5a1839a6ee6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:24:37.362910 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7b6f38-0bcd-4eba-9344-a5a1839a6ee6 service cinder] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:24:37.377035 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 824/1650] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:24:37 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:24:37.822893 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7803f444-6418-4c67-8f4e-ee7667d1a802 service cinder] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:24:37.832342 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7b6f38-0bcd-4eba-9344-a5a1839a6ee6 service cinder] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:24:37.832342 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7b6f38-0bcd-4eba-9344-a5a1839a6ee6 service cinder] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:24:37.832342 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc5e37b-e47f-4bc2-827e-d8ae87bd61ff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff7931b75a924394b1af733febce0834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:24:37.832342 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc5e37b-e47f-4bc2-827e-d8ae87bd61ff service cinder] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:37.837423 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc5e37b-e47f-4bc2-827e-d8ae87bd61ff service cinder] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:37.837423 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc5e37b-e47f-4bc2-827e-d8ae87bd61ff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:37.855196 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7803f444-6418-4c67-8f4e-ee7667d1a802 service cinder] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:24:37.873722 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9c01c1-3f38-4dca-a016-d26215448f94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:24:37.873722 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9c01c1-3f38-4dca-a016-d26215448f94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:37.873722 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9c01c1-3f38-4dca-a016-d26215448f94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:37.873722 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9c01c1-3f38-4dca-a016-d26215448f94 admin admin] PATH_INFO: `/v3/users/75e79f6440b84ab2a9de551f2ff5b224` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:37.875069 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9c01c1-3f38-4dca-a016-d26215448f94 admin admin] RBAC: Authorizing `identity:delete_user(user_id=75e79f6440b84ab2a9de551f2ff5b224)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:24:37.877965 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: WARNING py.warnings [None req-1b9c01c1-3f38-4dca-a016-d26215448f94 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 29 11:24:37.877965 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: warnings.warn(msg) Jun 29 11:24:37.877965 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: Jun 29 11:24:37.881016 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9c01c1-3f38-4dca-a016-d26215448f94 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:24:37.881136 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc5e37b-e47f-4bc2-827e-d8ae87bd61ff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:24:37.885257 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc5e37b-e47f-4bc2-827e-d8ae87bd61ff service cinder] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:24:37.903664 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 825/1651] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:24:37 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:24:37.904802 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc5e37b-e47f-4bc2-827e-d8ae87bd61ff service cinder] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:24:37.941808 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc5e37b-e47f-4bc2-827e-d8ae87bd61ff service cinder] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:24:37.957237 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9c3a8e-736d-46fa-aab2-9b634a916210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:24:37.960444 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9c3a8e-736d-46fa-aab2-9b634a916210 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:37.966798 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9c3a8e-736d-46fa-aab2-9b634a916210 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:37.973066 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9c3a8e-736d-46fa-aab2-9b634a916210 admin admin] PATH_INFO: `/v3/users/61eadda6693f47b680fb2083bf8bd573` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:37.978607 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9c3a8e-736d-46fa-aab2-9b634a916210 admin admin] RBAC: Authorizing `identity:delete_user(user_id=61eadda6693f47b680fb2083bf8bd573)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:24:37.984668 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9c3a8e-736d-46fa-aab2-9b634a916210 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:24:38.230586 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-1b9c01c1-3f38-4dca-a016-d26215448f94 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '75e79f6440b84ab2a9de551f2ff5b224'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:24:38.312148 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-1e9c3a8e-736d-46fa-aab2-9b634a916210 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '61eadda6693f47b680fb2083bf8bd573'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:24:38.313481 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-1e9c3a8e-736d-46fa-aab2-9b634a916210 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '61eadda6693f47b680fb2083bf8bd573'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:24:38.323319 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-1b9c01c1-3f38-4dca-a016-d26215448f94 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '75e79f6440b84ab2a9de551f2ff5b224'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:24:38.324006 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-1b9c01c1-3f38-4dca-a016-d26215448f94 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '75e79f6440b84ab2a9de551f2ff5b224'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:24:38.378063 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-1b9c01c1-3f38-4dca-a016-d26215448f94 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '75e79f6440b84ab2a9de551f2ff5b224'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:24:38.378815 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-1b9c01c1-3f38-4dca-a016-d26215448f94 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '75e79f6440b84ab2a9de551f2ff5b224'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:24:38.419351 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 827/1652] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jun 29 11:24:37 2022] DELETE /identity/v3/users/75e79f6440b84ab2a9de551f2ff5b224 => generated 0 bytes in 735 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:24:38.420246 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9c01c1-3f38-4dca-a016-d26215448f94 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:24:38.424309 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-1b9c01c1-3f38-4dca-a016-d26215448f94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:24:38.447160 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-1e9c3a8e-736d-46fa-aab2-9b634a916210 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '61eadda6693f47b680fb2083bf8bd573'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:24:38.462320 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-1e9c3a8e-736d-46fa-aab2-9b634a916210 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '61eadda6693f47b680fb2083bf8bd573'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:24:38.465248 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-1e9c3a8e-736d-46fa-aab2-9b634a916210 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '61eadda6693f47b680fb2083bf8bd573'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:24:38.511434 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 826/1653] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jun 29 11:24:37 2022] DELETE /identity/v3/users/61eadda6693f47b680fb2083bf8bd573 => generated 0 bytes in 604 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:24:38.511434 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84f4d7a-f710-488b-a70f-492f78dffb06 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:38.511434 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84f4d7a-f710-488b-a70f-492f78dffb06 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:38.511434 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84f4d7a-f710-488b-a70f-492f78dffb06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:38.521702 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9c01c1-3f38-4dca-a016-d26215448f94 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:24:38.570386 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-c84f4d7a-f710-488b-a70f-492f78dffb06 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:24:38.571891 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-c84f4d7a-f710-488b-a70f-492f78dffb06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:24:38.645953 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da37d9b5-758f-40c5-8bf0-460eee4ecae7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff7931b75a924394b1af733febce0834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:24:38.648275 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da37d9b5-758f-40c5-8bf0-460eee4ecae7 service cinder] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:38.650052 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da37d9b5-758f-40c5-8bf0-460eee4ecae7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:38.650866 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da37d9b5-758f-40c5-8bf0-460eee4ecae7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:38.659311 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-da37d9b5-758f-40c5-8bf0-460eee4ecae7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:24:38.785502 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da37d9b5-758f-40c5-8bf0-460eee4ecae7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:24:38.787707 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da37d9b5-758f-40c5-8bf0-460eee4ecae7 service cinder] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:24:38.790743 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:24:38.818869 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 828/1654] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:24:38 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:24:38.823569 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e890ee6d-cbe2-4908-b696-d93ebf66044e None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:38.824162 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e890ee6d-cbe2-4908-b696-d93ebf66044e None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:38.824714 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e890ee6d-cbe2-4908-b696-d93ebf66044e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:38.847642 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 827/1655] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:24:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 338 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:24:38.853193 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:24:38.877728 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-e890ee6d-cbe2-4908-b696-d93ebf66044e None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:24:38.879209 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-e890ee6d-cbe2-4908-b696-d93ebf66044e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:24:38.900845 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 829/1656] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:24:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:24:38.919966 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84f4d7a-f710-488b-a70f-492f78dffb06 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:24:38.922197 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-c84f4d7a-f710-488b-a70f-492f78dffb06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:24:38.930521 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e890ee6d-cbe2-4908-b696-d93ebf66044e None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:24:38.932469 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-e890ee6d-cbe2-4908-b696-d93ebf66044e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:24:39.071692 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84f4d7a-f710-488b-a70f-492f78dffb06 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:24:39.077285 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4494198b-2065-46c3-98ae-3ab51f383b2e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f4bb9f989e4024a5da28dced646b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:24:39.078238 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4494198b-2065-46c3-98ae-3ab51f383b2e service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:39.078665 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4494198b-2065-46c3-98ae-3ab51f383b2e service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:39.079084 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4494198b-2065-46c3-98ae-3ab51f383b2e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:39.103402 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4494198b-2065-46c3-98ae-3ab51f383b2e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:24:39.109082 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4494198b-2065-46c3-98ae-3ab51f383b2e service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:24:39.117804 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e890ee6d-cbe2-4908-b696-d93ebf66044e None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:24:39.127150 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06562ec1-0cd2-491b-ae8e-fde156c868bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f4bb9f989e4024a5da28dced646b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:24:39.129165 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06562ec1-0cd2-491b-ae8e-fde156c868bb service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:39.130495 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06562ec1-0cd2-491b-ae8e-fde156c868bb service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:39.131794 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06562ec1-0cd2-491b-ae8e-fde156c868bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:39.140204 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 828/1657] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:24:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:24:39.154566 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06562ec1-0cd2-491b-ae8e-fde156c868bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:24:39.160508 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06562ec1-0cd2-491b-ae8e-fde156c868bb service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:24:39.169965 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 830/1658] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:24:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:24:40.938097 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4494198b-2065-46c3-98ae-3ab51f383b2e service neutron] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:24:40.939219 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-4494198b-2065-46c3-98ae-3ab51f383b2e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:24:40.949221 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06562ec1-0cd2-491b-ae8e-fde156c868bb service neutron] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:24:40.951632 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-06562ec1-0cd2-491b-ae8e-fde156c868bb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:24:40.992141 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06562ec1-0cd2-491b-ae8e-fde156c868bb service neutron] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:24:41.002995 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4494198b-2065-46c3-98ae-3ab51f383b2e service neutron] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:24:41.037145 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2529cd5-f3c8-45f5-bc31-73fb3a615ff2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:24:41.042362 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569a5ebf-1b6b-4547-8db7-030bbf45cb0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:24:41.044012 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2529cd5-f3c8-45f5-bc31-73fb3a615ff2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:41.045092 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569a5ebf-1b6b-4547-8db7-030bbf45cb0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:41.048263 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2529cd5-f3c8-45f5-bc31-73fb3a615ff2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:41.048718 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2529cd5-f3c8-45f5-bc31-73fb3a615ff2 admin admin] PATH_INFO: `/v3/projects/db854603a9334b68a6a482ba2bcfc0b9` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:41.049187 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569a5ebf-1b6b-4547-8db7-030bbf45cb0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:41.050288 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569a5ebf-1b6b-4547-8db7-030bbf45cb0b admin admin] PATH_INFO: `/v3/projects/6815b6ed40654b9cb7ab3567b3f752a1` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:41.052558 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569a5ebf-1b6b-4547-8db7-030bbf45cb0b admin admin] RBAC: Authorizing `identity:delete_project(project_id=6815b6ed40654b9cb7ab3567b3f752a1)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:24:41.055978 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2529cd5-f3c8-45f5-bc31-73fb3a615ff2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=db854603a9334b68a6a482ba2bcfc0b9)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:24:41.058148 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2529cd5-f3c8-45f5-bc31-73fb3a615ff2 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:24:41.063304 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: WARNING py.warnings [None req-569a5ebf-1b6b-4547-8db7-030bbf45cb0b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 29 11:24:41.063304 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: warnings.warn(msg) Jun 29 11:24:41.063304 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: Jun 29 11:24:41.064553 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569a5ebf-1b6b-4547-8db7-030bbf45cb0b admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:24:41.169487 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:24:41.284451 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:24:41.298550 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-c2529cd5-f3c8-45f5-bc31-73fb3a615ff2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db854603a9334b68a6a482ba2bcfc0b9', 'request_id': 'req-c2529cd5-f3c8-45f5-bc31-73fb3a615ff2'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:24:41.376431 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-569a5ebf-1b6b-4547-8db7-030bbf45cb0b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6815b6ed40654b9cb7ab3567b3f752a1', 'request_id': 'req-569a5ebf-1b6b-4547-8db7-030bbf45cb0b'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:24:41.418557 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-c2529cd5-f3c8-45f5-bc31-73fb3a615ff2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db854603a9334b68a6a482ba2bcfc0b9', 'request_id': 'req-c2529cd5-f3c8-45f5-bc31-73fb3a615ff2'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:24:41.423208 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-c2529cd5-f3c8-45f5-bc31-73fb3a615ff2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db854603a9334b68a6a482ba2bcfc0b9', 'request_id': 'req-c2529cd5-f3c8-45f5-bc31-73fb3a615ff2'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:24:41.437043 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-569a5ebf-1b6b-4547-8db7-030bbf45cb0b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6815b6ed40654b9cb7ab3567b3f752a1', 'request_id': 'req-569a5ebf-1b6b-4547-8db7-030bbf45cb0b'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:24:41.466830 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-c2529cd5-f3c8-45f5-bc31-73fb3a615ff2 admin admin] The token cache is being invalidate because project db854603a9334b68a6a482ba2bcfc0b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 11:24:41.467099 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-c2529cd5-f3c8-45f5-bc31-73fb3a615ff2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:24:41.473373 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-c2529cd5-f3c8-45f5-bc31-73fb3a615ff2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:24:41.473373 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 831/1659] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jun 29 11:24:40 2022] DELETE /identity/v3/projects/db854603a9334b68a6a482ba2bcfc0b9 => generated 0 bytes in 525 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 11:24:41.480889 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2529cd5-f3c8-45f5-bc31-73fb3a615ff2 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:24:41.487176 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-c2529cd5-f3c8-45f5-bc31-73fb3a615ff2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:24:41.502596 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-569a5ebf-1b6b-4547-8db7-030bbf45cb0b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6815b6ed40654b9cb7ab3567b3f752a1', 'request_id': 'req-569a5ebf-1b6b-4547-8db7-030bbf45cb0b'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:24:41.618580 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-569a5ebf-1b6b-4547-8db7-030bbf45cb0b admin admin] The token cache is being invalidate because project 6815b6ed40654b9cb7ab3567b3f752a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 11:24:41.619171 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-569a5ebf-1b6b-4547-8db7-030bbf45cb0b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:24:41.620028 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-569a5ebf-1b6b-4547-8db7-030bbf45cb0b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:24:41.625863 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 829/1660] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jun 29 11:24:40 2022] DELETE /identity/v3/projects/6815b6ed40654b9cb7ab3567b3f752a1 => generated 0 bytes in 686 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:24:41.629928 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc8ab44-06d1-47e7-a886-bb08cee00390 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:41.630836 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc8ab44-06d1-47e7-a886-bb08cee00390 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:41.631600 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc8ab44-06d1-47e7-a886-bb08cee00390 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:41.690761 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-0bc8ab44-06d1-47e7-a886-bb08cee00390 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:24:41.692533 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-0bc8ab44-06d1-47e7-a886-bb08cee00390 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:24:41.722265 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2529cd5-f3c8-45f5-bc31-73fb3a615ff2 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:24:41.858281 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 830/1661] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:24:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:24:41.865740 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7832c4ce-30c3-453d-9afe-74c70b864451 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:41.866238 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7832c4ce-30c3-453d-9afe-74c70b864451 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:41.866733 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7832c4ce-30c3-453d-9afe-74c70b864451 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:41.886041 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d86203-940f-4552-8222-81419a2f8d6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:24:41.887243 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d86203-940f-4552-8222-81419a2f8d6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:41.887689 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d86203-940f-4552-8222-81419a2f8d6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:41.888142 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d86203-940f-4552-8222-81419a2f8d6b admin admin] PATH_INFO: `/v3/users/7851a39b35a442f0b062de78adf7fead` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:41.892021 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d86203-940f-4552-8222-81419a2f8d6b admin admin] RBAC: Authorizing `identity:delete_user(user_id=7851a39b35a442f0b062de78adf7fead)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:24:41.893419 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d86203-940f-4552-8222-81419a2f8d6b admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:24:41.907866 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-7832c4ce-30c3-453d-9afe-74c70b864451 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:24:41.910692 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-7832c4ce-30c3-453d-9afe-74c70b864451 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:24:41.934811 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 831/1662] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:24:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 11:24:41.938205 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7832c4ce-30c3-453d-9afe-74c70b864451 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:24:41.955444 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7832c4ce-30c3-453d-9afe-74c70b864451 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:24:41.962963 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a99622-466b-43fe-a1ab-c8255236678d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:24:41.964025 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a99622-466b-43fe-a1ab-c8255236678d admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:41.964583 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a99622-466b-43fe-a1ab-c8255236678d admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:41.965015 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a99622-466b-43fe-a1ab-c8255236678d admin admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:41.972289 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a99622-466b-43fe-a1ab-c8255236678d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:24:41.972289 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a99622-466b-43fe-a1ab-c8255236678d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:24:41.972289 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a99622-466b-43fe-a1ab-c8255236678d admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:24:41.986551 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 832/1663] 10.222.0.44 () {58 vars in 1152 bytes} [Wed Jun 29 11:24:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:24:41.990133 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a99622-466b-43fe-a1ab-c8255236678d admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:24:42.009356 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a99622-466b-43fe-a1ab-c8255236678d admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:24:42.016585 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033229f5-6c4a-44e9-a633-e17278704a1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:24:42.017657 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033229f5-6c4a-44e9-a633-e17278704a1e admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:42.018364 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033229f5-6c4a-44e9-a633-e17278704a1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:42.018835 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033229f5-6c4a-44e9-a633-e17278704a1e admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:42.019729 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033229f5-6c4a-44e9-a633-e17278704a1e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:24:42.022050 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033229f5-6c4a-44e9-a633-e17278704a1e admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:24:42.142144 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-45d86203-940f-4552-8222-81419a2f8d6b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7851a39b35a442f0b062de78adf7fead'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:24:42.188653 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 833/1664] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jun 29 11:24:41 2022] POST /identity/v3/projects => generated 357 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:24:42.192127 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033229f5-6c4a-44e9-a633-e17278704a1e admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:24:42.234822 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-45d86203-940f-4552-8222-81419a2f8d6b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7851a39b35a442f0b062de78adf7fead'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:24:42.235573 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-45d86203-940f-4552-8222-81419a2f8d6b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7851a39b35a442f0b062de78adf7fead'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:24:42.273153 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-45d86203-940f-4552-8222-81419a2f8d6b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7851a39b35a442f0b062de78adf7fead'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:24:42.274037 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-45d86203-940f-4552-8222-81419a2f8d6b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7851a39b35a442f0b062de78adf7fead'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:24:42.314655 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 832/1665] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jun 29 11:24:41 2022] DELETE /identity/v3/users/7851a39b35a442f0b062de78adf7fead => generated 0 bytes in 831 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:24:42.390474 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033229f5-6c4a-44e9-a633-e17278704a1e admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:24:42.567466 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12efa437-be13-440f-bae1-18033be3ac06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:24:42.568903 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12efa437-be13-440f-bae1-18033be3ac06 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:42.572005 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12efa437-be13-440f-bae1-18033be3ac06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:42.572348 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12efa437-be13-440f-bae1-18033be3ac06 admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:42.579271 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12efa437-be13-440f-bae1-18033be3ac06 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:24:42.581816 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12efa437-be13-440f-bae1-18033be3ac06 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:24:42.739418 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 834/1666] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jun 29 11:24:42 2022] POST /identity/v3/users => generated 279 bytes in 548 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:24:42.743284 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d86203-940f-4552-8222-81419a2f8d6b admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:24:42.745811 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-45d86203-940f-4552-8222-81419a2f8d6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:24:42.765579 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:24:42.771073 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d86203-940f-4552-8222-81419a2f8d6b admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:24:42.778526 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b677865-e2a9-49a7-b912-105633e04aea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:24:42.781022 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b677865-e2a9-49a7-b912-105633e04aea admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:42.781507 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b677865-e2a9-49a7-b912-105633e04aea admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:42.782365 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b677865-e2a9-49a7-b912-105633e04aea admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:42.783332 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b677865-e2a9-49a7-b912-105633e04aea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:24:42.795420 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b677865-e2a9-49a7-b912-105633e04aea admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:24:42.816806 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 833/1667] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:24:42 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 11:24:42.818495 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12efa437-be13-440f-bae1-18033be3ac06 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:24:42.832781 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12efa437-be13-440f-bae1-18033be3ac06 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:24:42.837723 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a36719-634a-4f85-8c91-4c67b104d0fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:24:42.838514 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a36719-634a-4f85-8c91-4c67b104d0fb admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:42.838924 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a36719-634a-4f85-8c91-4c67b104d0fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:42.839307 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a36719-634a-4f85-8c91-4c67b104d0fb admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:42.840845 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a36719-634a-4f85-8c91-4c67b104d0fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:24:42.842828 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a36719-634a-4f85-8c91-4c67b104d0fb admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:24:42.859108 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 835/1668] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:24:42 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 11:24:42.863855 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b677865-e2a9-49a7-b912-105633e04aea admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:24:42.885876 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b677865-e2a9-49a7-b912-105633e04aea admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:24:42.894336 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cfe5a5-edab-4f83-9bde-e7dd96d0b349 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:24:42.895636 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cfe5a5-edab-4f83-9bde-e7dd96d0b349 admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:42.896254 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cfe5a5-edab-4f83-9bde-e7dd96d0b349 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:42.896759 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cfe5a5-edab-4f83-9bde-e7dd96d0b349 admin admin] PATH_INFO: `/v3/projects/05d01361b1924cd4a77fb6c8bd887c58/users/f89a284f25484d1b92f4003091a5638d/roles/e1c88eebe911432cab6a07fbeb84653b` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:42.927610 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cfe5a5-edab-4f83-9bde-e7dd96d0b349 admin admin] RBAC: Authorizing `identity:create_grant(project_id=05d01361b1924cd4a77fb6c8bd887c58, user_id=f89a284f25484d1b92f4003091a5638d, role_id=e1c88eebe911432cab6a07fbeb84653b)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:24:42.931627 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cfe5a5-edab-4f83-9bde-e7dd96d0b349 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:24:43.047489 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 834/1669] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jun 29 11:24:42 2022] PUT /identity/v3/projects/05d01361b1924cd4a77fb6c8bd887c58/users/f89a284f25484d1b92f4003091a5638d/roles/e1c88eebe911432cab6a07fbeb84653b => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 11:24:43.058720 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb8c139-0db5-442d-b5d3-6c6993fcd19d None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:43.059209 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb8c139-0db5-442d-b5d3-6c6993fcd19d None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:43.059512 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb8c139-0db5-442d-b5d3-6c6993fcd19d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:43.104132 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-abb8c139-0db5-442d-b5d3-6c6993fcd19d None None] MFA Rules not processed for user `f89a284f25484d1b92f4003091a5638d`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:24:43.106502 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-abb8c139-0db5-442d-b5d3-6c6993fcd19d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:24:43.260907 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 836/1670] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:24:43 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:24:43.265061 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cfe5a5-edab-4f83-9bde-e7dd96d0b349 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:24:43.308460 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e14edf-94bc-4057-8e2a-1379308fe003 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:43.308753 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e14edf-94bc-4057-8e2a-1379308fe003 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:43.309033 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e14edf-94bc-4057-8e2a-1379308fe003 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:43.363405 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-58e14edf-94bc-4057-8e2a-1379308fe003 None None] MFA Rules not processed for user `b1ca7ff32b434abc8d036ddfa266a584`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:24:43.364697 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-58e14edf-94bc-4057-8e2a-1379308fe003 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:24:43.438297 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cfe5a5-edab-4f83-9bde-e7dd96d0b349 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:24:43.455619 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa60423d-96c9-416f-99c5-ae1e68597f72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:24:43.456583 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa60423d-96c9-416f-99c5-ae1e68597f72 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:43.457370 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa60423d-96c9-416f-99c5-ae1e68597f72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:43.457789 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa60423d-96c9-416f-99c5-ae1e68597f72 admin admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:43.458743 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa60423d-96c9-416f-99c5-ae1e68597f72 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:24:43.479731 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa60423d-96c9-416f-99c5-ae1e68597f72 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:24:43.525233 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 837/1671] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:24:43 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:24:43.529846 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1029ab-d522-4bf0-8da1-5af3080ed845 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:43.535593 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1029ab-d522-4bf0-8da1-5af3080ed845 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:43.537750 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1029ab-d522-4bf0-8da1-5af3080ed845 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:43.573607 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-dc1029ab-d522-4bf0-8da1-5af3080ed845 None None] MFA Rules not processed for user `b1ca7ff32b434abc8d036ddfa266a584`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:24:43.574864 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-dc1029ab-d522-4bf0-8da1-5af3080ed845 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:24:43.599202 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 838/1672] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:24:43 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:24:43.609450 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1029ab-d522-4bf0-8da1-5af3080ed845 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:24:43.611271 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-dc1029ab-d522-4bf0-8da1-5af3080ed845 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:24:43.616815 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 835/1673] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jun 29 11:24:43 2022] POST /identity/v3/projects => generated 353 bytes in 352 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:24:43.620848 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa60423d-96c9-416f-99c5-ae1e68597f72 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:24:43.679619 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:24:43.784650 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1029ab-d522-4bf0-8da1-5af3080ed845 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:24:43.840293 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa60423d-96c9-416f-99c5-ae1e68597f72 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:24:43.853404 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e372d869-1a96-4c03-9375-cbcccdc08bcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:24:43.854362 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e372d869-1a96-4c03-9375-cbcccdc08bcf admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:43.854836 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e372d869-1a96-4c03-9375-cbcccdc08bcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:43.855249 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e372d869-1a96-4c03-9375-cbcccdc08bcf admin admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:43.856160 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e372d869-1a96-4c03-9375-cbcccdc08bcf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:24:43.858515 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e372d869-1a96-4c03-9375-cbcccdc08bcf admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:24:43.941155 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39ece46-70df-4a95-821c-84a23f8394b6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '796aeeff16df4739bddec1f6dee61bfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:24:43.946111 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39ece46-70df-4a95-821c-84a23f8394b6 service nova] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:43.946616 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39ece46-70df-4a95-821c-84a23f8394b6 service nova] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:43.949116 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39ece46-70df-4a95-821c-84a23f8394b6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:43.992945 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 836/1674] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jun 29 11:24:43 2022] POST /identity/v3/users => generated 283 bytes in 372 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:24:43.995237 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e372d869-1a96-4c03-9375-cbcccdc08bcf admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:24:43.998692 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-e372d869-1a96-4c03-9375-cbcccdc08bcf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:24:44.069264 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39ece46-70df-4a95-821c-84a23f8394b6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:24:44.071954 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39ece46-70df-4a95-821c-84a23f8394b6 service nova] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:24:44.073463 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e372d869-1a96-4c03-9375-cbcccdc08bcf admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:24:44.083463 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f20d3f-d97f-49e6-812a-351a2c128b5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:24:44.088057 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f20d3f-d97f-49e6-812a-351a2c128b5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:44.088517 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f20d3f-d97f-49e6-812a-351a2c128b5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:44.089380 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f20d3f-d97f-49e6-812a-351a2c128b5b admin admin] PATH_INFO: `/v3/projects/d8508b04f40947389176c04e55368fb4` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:44.091745 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f20d3f-d97f-49e6-812a-351a2c128b5b admin admin] RBAC: Authorizing `identity:delete_project(project_id=d8508b04f40947389176c04e55368fb4)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:24:44.094800 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f20d3f-d97f-49e6-812a-351a2c128b5b admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:24:44.110987 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 839/1675] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:24:43 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 502 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:24:44.113894 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39ece46-70df-4a95-821c-84a23f8394b6 service nova] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:24:44.142423 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39ece46-70df-4a95-821c-84a23f8394b6 service nova] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:24:44.154148 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968b135a-0682-465b-a5b0-139b4bf5e735 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:24:44.154902 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968b135a-0682-465b-a5b0-139b4bf5e735 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:44.155195 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968b135a-0682-465b-a5b0-139b4bf5e735 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:44.155573 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968b135a-0682-465b-a5b0-139b4bf5e735 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:44.156348 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968b135a-0682-465b-a5b0-139b4bf5e735 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:24:44.163760 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968b135a-0682-465b-a5b0-139b4bf5e735 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:24:44.185254 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 840/1676] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:24:44 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 11:24:44.188263 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968b135a-0682-465b-a5b0-139b4bf5e735 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:24:44.210519 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968b135a-0682-465b-a5b0-139b4bf5e735 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:24:44.243953 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8179dacd-7f35-4420-88a7-4b6f0887c0f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:24:44.248240 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8179dacd-7f35-4420-88a7-4b6f0887c0f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:44.249583 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8179dacd-7f35-4420-88a7-4b6f0887c0f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:44.250567 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8179dacd-7f35-4420-88a7-4b6f0887c0f9 admin admin] PATH_INFO: `/v3/projects/d80466559f8249c0996641dce576e034/users/fcf645ee87d5473db4a4a57533dc7c52/roles/e95c2ce63e114cc19a9dda6ad21e794f` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:44.265968 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-d7f20d3f-d97f-49e6-812a-351a2c128b5b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd8508b04f40947389176c04e55368fb4', 'request_id': 'req-d7f20d3f-d97f-49e6-812a-351a2c128b5b'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:24:44.286399 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8179dacd-7f35-4420-88a7-4b6f0887c0f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d80466559f8249c0996641dce576e034, user_id=fcf645ee87d5473db4a4a57533dc7c52, role_id=e95c2ce63e114cc19a9dda6ad21e794f)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:24:44.290140 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8179dacd-7f35-4420-88a7-4b6f0887c0f9 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:24:44.381823 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-d7f20d3f-d97f-49e6-812a-351a2c128b5b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd8508b04f40947389176c04e55368fb4', 'request_id': 'req-d7f20d3f-d97f-49e6-812a-351a2c128b5b'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:24:44.396304 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-d7f20d3f-d97f-49e6-812a-351a2c128b5b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd8508b04f40947389176c04e55368fb4', 'request_id': 'req-d7f20d3f-d97f-49e6-812a-351a2c128b5b'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:24:44.410127 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 841/1677] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jun 29 11:24:44 2022] PUT /identity/v3/projects/d80466559f8249c0996641dce576e034/users/fcf645ee87d5473db4a4a57533dc7c52/roles/e95c2ce63e114cc19a9dda6ad21e794f => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:24:44.413760 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8179dacd-7f35-4420-88a7-4b6f0887c0f9 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:24:44.422858 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-d7f20d3f-d97f-49e6-812a-351a2c128b5b admin admin] The token cache is being invalidate because project d8508b04f40947389176c04e55368fb4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 11:24:44.423712 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-d7f20d3f-d97f-49e6-812a-351a2c128b5b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:24:44.425829 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-d7f20d3f-d97f-49e6-812a-351a2c128b5b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:24:44.437583 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 837/1678] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jun 29 11:24:43 2022] DELETE /identity/v3/projects/d8508b04f40947389176c04e55368fb4 => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:24:44.439603 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9720fdee-4c07-4182-b9c4-714614a63721 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:44.440827 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9720fdee-4c07-4182-b9c4-714614a63721 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:44.441447 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9720fdee-4c07-4182-b9c4-714614a63721 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:44.472889 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-9720fdee-4c07-4182-b9c4-714614a63721 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:24:44.481330 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-9720fdee-4c07-4182-b9c4-714614a63721 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:24:44.613368 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=94,keystone:DELETE=1,keystone:INSERT=5 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:24:44.624750 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8179dacd-7f35-4420-88a7-4b6f0887c0f9 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:24:44.641131 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 838/1679] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:24:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:24:44.643194 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096e411f-d31c-4da5-b796-08f958f55579 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:44.644069 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096e411f-d31c-4da5-b796-08f958f55579 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:44.644069 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096e411f-d31c-4da5-b796-08f958f55579 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:44.644254 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4a9301-259a-4b36-b379-8b2f4ae0c2f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:24:44.644909 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4a9301-259a-4b36-b379-8b2f4ae0c2f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:44.645727 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4a9301-259a-4b36-b379-8b2f4ae0c2f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:44.646907 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4a9301-259a-4b36-b379-8b2f4ae0c2f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:44.654677 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4a9301-259a-4b36-b379-8b2f4ae0c2f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:24:44.657940 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4a9301-259a-4b36-b379-8b2f4ae0c2f3 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:24:44.674476 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-096e411f-d31c-4da5-b796-08f958f55579 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:24:44.675691 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-096e411f-d31c-4da5-b796-08f958f55579 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:24:44.682008 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 842/1680] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:24:44 2022] GET /identity/v3/roles => generated 1420 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 11:24:44.683452 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4a9301-259a-4b36-b379-8b2f4ae0c2f3 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:24:44.686141 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-7c4a9301-259a-4b36-b379-8b2f4ae0c2f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:24:44.704274 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 839/1681] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:24:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 11:24:44.706013 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096e411f-d31c-4da5-b796-08f958f55579 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:24:44.712068 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4a9301-259a-4b36-b379-8b2f4ae0c2f3 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:24:44.724012 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ab6e66-c189-4d6c-9d84-e7804ae005c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:24:44.724933 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ab6e66-c189-4d6c-9d84-e7804ae005c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:44.725221 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ab6e66-c189-4d6c-9d84-e7804ae005c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:44.725494 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ab6e66-c189-4d6c-9d84-e7804ae005c9 admin admin] PATH_INFO: `/v3/projects/d80466559f8249c0996641dce576e034/users/fcf645ee87d5473db4a4a57533dc7c52/roles/e95c2ce63e114cc19a9dda6ad21e794f` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:44.728719 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096e411f-d31c-4da5-b796-08f958f55579 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:24:44.730782 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ab6e66-c189-4d6c-9d84-e7804ae005c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d80466559f8249c0996641dce576e034, user_id=fcf645ee87d5473db4a4a57533dc7c52, role_id=e95c2ce63e114cc19a9dda6ad21e794f)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:24:44.733487 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ab6e66-c189-4d6c-9d84-e7804ae005c9 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:24:44.734602 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce3a74f-132a-484c-8ae3-5189de27dba2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:24:44.735490 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce3a74f-132a-484c-8ae3-5189de27dba2 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:44.736038 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce3a74f-132a-484c-8ae3-5189de27dba2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:44.743297 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce3a74f-132a-484c-8ae3-5189de27dba2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:44.745655 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce3a74f-132a-484c-8ae3-5189de27dba2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:24:44.746222 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce3a74f-132a-484c-8ae3-5189de27dba2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:24:44.748460 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce3a74f-132a-484c-8ae3-5189de27dba2 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:24:44.771856 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 843/1682] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jun 29 11:24:44 2022] PUT /identity/v3/projects/d80466559f8249c0996641dce576e034/users/fcf645ee87d5473db4a4a57533dc7c52/roles/e95c2ce63e114cc19a9dda6ad21e794f => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:24:44.774450 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bce1aa2-19cf-48e7-bdc5-18786ca27021 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:44.774450 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bce1aa2-19cf-48e7-bdc5-18786ca27021 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:44.774450 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bce1aa2-19cf-48e7-bdc5-18786ca27021 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:44.790581 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 840/1683] 10.222.0.44 () {58 vars in 1152 bytes} [Wed Jun 29 11:24:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:24:44.793750 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce3a74f-132a-484c-8ae3-5189de27dba2 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:24:44.823435 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce3a74f-132a-484c-8ae3-5189de27dba2 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:24:44.837178 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-2bce1aa2-19cf-48e7-bdc5-18786ca27021 None None] MFA Rules not processed for user `fcf645ee87d5473db4a4a57533dc7c52`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:24:44.840903 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-2bce1aa2-19cf-48e7-bdc5-18786ca27021 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:24:44.997532 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d47a3d-ec98-41bf-a147-74cf407f88b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:24:44.998224 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d47a3d-ec98-41bf-a147-74cf407f88b9 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:44.998509 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d47a3d-ec98-41bf-a147-74cf407f88b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:44.998788 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d47a3d-ec98-41bf-a147-74cf407f88b9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:45.006810 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d47a3d-ec98-41bf-a147-74cf407f88b9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:24:45.013171 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d47a3d-ec98-41bf-a147-74cf407f88b9 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:24:45.045665 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 844/1684] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:24:44 2022] POST /identity/v3/auth/tokens => generated 3287 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:24:45.062782 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bce1aa2-19cf-48e7-bdc5-18786ca27021 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:24:45.064882 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-2bce1aa2-19cf-48e7-bdc5-18786ca27021 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:24:45.160315 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 841/1685] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jun 29 11:24:44 2022] POST /identity/v3/projects => generated 391 bytes in 363 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:24:45.163640 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8413cc8-ce94-4a01-aee8-4f56ede96cc4 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:45.163640 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8413cc8-ce94-4a01-aee8-4f56ede96cc4 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:45.164740 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8413cc8-ce94-4a01-aee8-4f56ede96cc4 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:45.168923 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 842/1686] 10.222.0.44 () {54 vars in 912 bytes} [Wed Jun 29 11:24:45 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:24:45.168923 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8413cc8-ce94-4a01-aee8-4f56ede96cc4 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:24:45.173702 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bce1aa2-19cf-48e7-bdc5-18786ca27021 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:24:45.256223 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8413cc8-ce94-4a01-aee8-4f56ede96cc4 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:24:45.261953 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6df059e-4ec0-4e62-a49c-c6e025c271a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:24:45.262741 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6df059e-4ec0-4e62-a49c-c6e025c271a5 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:45.263049 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6df059e-4ec0-4e62-a49c-c6e025c271a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:45.263753 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6df059e-4ec0-4e62-a49c-c6e025c271a5 admin admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:45.265263 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6df059e-4ec0-4e62-a49c-c6e025c271a5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:24:45.266636 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6df059e-4ec0-4e62-a49c-c6e025c271a5 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:24:45.268578 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a4700b-c3a7-4a0a-83d4-a99f6482ef4d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff7931b75a924394b1af733febce0834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:24:45.269623 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a4700b-c3a7-4a0a-83d4-a99f6482ef4d service cinder] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:45.269623 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a4700b-c3a7-4a0a-83d4-a99f6482ef4d service cinder] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:45.269924 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a4700b-c3a7-4a0a-83d4-a99f6482ef4d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:45.361820 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:24:45.396048 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 843/1687] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jun 29 11:24:45 2022] POST /identity/v3/users => generated 296 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:24:45.397586 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd030b1-c0df-4667-bcfb-a74fdb5fa8f1 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:45.399203 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd030b1-c0df-4667-bcfb-a74fdb5fa8f1 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:45.399203 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd030b1-c0df-4667-bcfb-a74fdb5fa8f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:45.441892 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a4700b-c3a7-4a0a-83d4-a99f6482ef4d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:24:45.444709 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a4700b-c3a7-4a0a-83d4-a99f6482ef4d service cinder] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:24:45.491008 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 845/1688] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:24:45 2022] GET /identity/v3/auth/tokens => generated 3287 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:24:45.493117 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a4700b-c3a7-4a0a-83d4-a99f6482ef4d service cinder] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:24:45.530579 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a4700b-c3a7-4a0a-83d4-a99f6482ef4d service cinder] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:24:45.544479 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-1cd030b1-c0df-4667-bcfb-a74fdb5fa8f1 None None] MFA Rules not processed for user `b5f4bb9f989e4024a5da28dced646b92`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:24:45.550111 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cfb3da-836c-487c-83c8-b7058a04509d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:24:45.550731 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cfb3da-836c-487c-83c8-b7058a04509d admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:45.551014 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cfb3da-836c-487c-83c8-b7058a04509d admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:45.551288 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cfb3da-836c-487c-83c8-b7058a04509d admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:45.556160 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-1cd030b1-c0df-4667-bcfb-a74fdb5fa8f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:24:45.557132 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cfb3da-836c-487c-83c8-b7058a04509d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:24:45.559422 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cfb3da-836c-487c-83c8-b7058a04509d admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:24:45.602399 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 846/1689] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:24:45 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 11:24:45.606562 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cfb3da-836c-487c-83c8-b7058a04509d admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:24:45.652010 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cfb3da-836c-487c-83c8-b7058a04509d admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:24:45.659744 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39841e2-9245-41f2-837e-dd4636f3a933 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:24:45.660621 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39841e2-9245-41f2-837e-dd4636f3a933 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:45.660787 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39841e2-9245-41f2-837e-dd4636f3a933 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:45.661045 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39841e2-9245-41f2-837e-dd4636f3a933 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:45.661779 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39841e2-9245-41f2-837e-dd4636f3a933 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:24:45.664200 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39841e2-9245-41f2-837e-dd4636f3a933 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:24:45.679543 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 847/1690] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:24:45 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 11:24:45.684536 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39841e2-9245-41f2-837e-dd4636f3a933 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:24:45.705970 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39841e2-9245-41f2-837e-dd4636f3a933 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:24:45.721255 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec65903-2c41-4f28-b43d-d9b836e160df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:24:45.722183 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec65903-2c41-4f28-b43d-d9b836e160df admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:45.722471 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec65903-2c41-4f28-b43d-d9b836e160df admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:45.722745 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec65903-2c41-4f28-b43d-d9b836e160df admin admin] PATH_INFO: `/v3/projects/e2190591a56948758833f8895fbdc3b8/users/1e4c45395e3445c583b7c8ce55c61d83/roles/e1c88eebe911432cab6a07fbeb84653b` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:45.730828 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 844/1691] 10.222.0.44 () {58 vars in 1009 bytes} [Wed Jun 29 11:24:45 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 334 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:24:45.759484 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec65903-2c41-4f28-b43d-d9b836e160df admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2190591a56948758833f8895fbdc3b8, user_id=1e4c45395e3445c583b7c8ce55c61d83, role_id=e1c88eebe911432cab6a07fbeb84653b)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:24:45.779010 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec65903-2c41-4f28-b43d-d9b836e160df admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:24:45.797447 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd030b1-c0df-4667-bcfb-a74fdb5fa8f1 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:24:45.816971 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd030b1-c0df-4667-bcfb-a74fdb5fa8f1 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:24:45.825686 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d5b003-5a7e-4efb-b1c5-fb2ef35369c6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff7931b75a924394b1af733febce0834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:24:45.826547 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d5b003-5a7e-4efb-b1c5-fb2ef35369c6 service cinder] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:45.826934 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d5b003-5a7e-4efb-b1c5-fb2ef35369c6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:45.827323 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d5b003-5a7e-4efb-b1c5-fb2ef35369c6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:45.829789 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-79d5b003-5a7e-4efb-b1c5-fb2ef35369c6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:24:45.957672 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 848/1692] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jun 29 11:24:45 2022] PUT /identity/v3/projects/e2190591a56948758833f8895fbdc3b8/users/1e4c45395e3445c583b7c8ce55c61d83/roles/e1c88eebe911432cab6a07fbeb84653b => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:24:45.959258 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d9cbc6-2258-4a96-a103-e100566ce6aa None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:45.959868 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d9cbc6-2258-4a96-a103-e100566ce6aa None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:45.959868 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d9cbc6-2258-4a96-a103-e100566ce6aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:46.015261 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d5b003-5a7e-4efb-b1c5-fb2ef35369c6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:24:46.017669 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d5b003-5a7e-4efb-b1c5-fb2ef35369c6 service cinder] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:24:46.029291 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 845/1693] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:24:45 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 11:24:46.030424 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d5b003-5a7e-4efb-b1c5-fb2ef35369c6 service cinder] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:24:46.033121 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-79d5b003-5a7e-4efb-b1c5-fb2ef35369c6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:24:46.074263 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d5b003-5a7e-4efb-b1c5-fb2ef35369c6 service cinder] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:24:46.081479 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-03d9cbc6-2258-4a96-a103-e100566ce6aa None None] MFA Rules not processed for user `1e4c45395e3445c583b7c8ce55c61d83`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:24:46.082933 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-03d9cbc6-2258-4a96-a103-e100566ce6aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:24:46.211980 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f7691a-beb9-471e-8c31-5ace9fcfe7b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f4bb9f989e4024a5da28dced646b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:24:46.228024 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f7691a-beb9-471e-8c31-5ace9fcfe7b0 service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:46.230611 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f7691a-beb9-471e-8c31-5ace9fcfe7b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:46.232112 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f7691a-beb9-471e-8c31-5ace9fcfe7b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:46.242957 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:24:46.264300 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f7691a-beb9-471e-8c31-5ace9fcfe7b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:24:46.266387 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f7691a-beb9-471e-8c31-5ace9fcfe7b0 service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:24:46.285695 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 846/1694] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:24:46 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:24:46.287556 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9733fa3-36df-49ea-ac13-0a509804d550 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:46.287990 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9733fa3-36df-49ea-ac13-0a509804d550 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:46.288176 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9733fa3-36df-49ea-ac13-0a509804d550 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:46.297414 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 849/1695] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:24:45 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 339 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:24:46.304466 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d9cbc6-2258-4a96-a103-e100566ce6aa None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:24:46.376634 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-d9733fa3-36df-49ea-ac13-0a509804d550 None None] MFA Rules not processed for user `f75e0b3528b14856b73dc441c52d1d73`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:24:46.378402 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-d9733fa3-36df-49ea-ac13-0a509804d550 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:24:46.458585 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d9cbc6-2258-4a96-a103-e100566ce6aa None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:24:46.473574 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4975720c-a285-48e7-a014-73617d906e34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:24:46.475193 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4975720c-a285-48e7-a014-73617d906e34 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:46.477717 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4975720c-a285-48e7-a014-73617d906e34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:46.478075 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4975720c-a285-48e7-a014-73617d906e34 admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:46.478920 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4975720c-a285-48e7-a014-73617d906e34 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:24:46.484738 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4975720c-a285-48e7-a014-73617d906e34 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:24:46.507687 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 847/1696] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:24:46 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:24:46.523048 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9733fa3-36df-49ea-ac13-0a509804d550 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:24:46.525306 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-d9733fa3-36df-49ea-ac13-0a509804d550 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:24:46.591007 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:24:46.608184 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 850/1697] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jun 29 11:24:46 2022] POST /identity/v3/projects => generated 389 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:24:46.612140 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4975720c-a285-48e7-a014-73617d906e34 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:24:46.791201 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9733fa3-36df-49ea-ac13-0a509804d550 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:24:46.841382 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4975720c-a285-48e7-a014-73617d906e34 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:24:46.856781 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55775296-9dca-4237-93d8-1eda2df4d691 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:24:46.861629 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55775296-9dca-4237-93d8-1eda2df4d691 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:46.862231 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55775296-9dca-4237-93d8-1eda2df4d691 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:46.862704 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55775296-9dca-4237-93d8-1eda2df4d691 admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:46.865063 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55775296-9dca-4237-93d8-1eda2df4d691 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:24:46.872146 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55775296-9dca-4237-93d8-1eda2df4d691 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:24:46.959213 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 851/1698] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jun 29 11:24:46 2022] POST /identity/v3/users => generated 301 bytes in 350 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:24:46.966083 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55775296-9dca-4237-93d8-1eda2df4d691 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:24:46.992021 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5d6d76-c54b-43fe-867f-53720ffa38fe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '796aeeff16df4739bddec1f6dee61bfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:24:46.998566 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55775296-9dca-4237-93d8-1eda2df4d691 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:24:46.998835 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5d6d76-c54b-43fe-867f-53720ffa38fe service nova] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:46.998835 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5d6d76-c54b-43fe-867f-53720ffa38fe service nova] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:46.998835 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5d6d76-c54b-43fe-867f-53720ffa38fe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:47.014699 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbedc32-4c7e-4f60-b96f-e717b20bc950 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:24:47.021436 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbedc32-4c7e-4f60-b96f-e717b20bc950 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:47.021969 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbedc32-4c7e-4f60-b96f-e717b20bc950 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:47.027200 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbedc32-4c7e-4f60-b96f-e717b20bc950 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:47.027200 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbedc32-4c7e-4f60-b96f-e717b20bc950 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:24:47.032806 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbedc32-4c7e-4f60-b96f-e717b20bc950 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:24:47.064603 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 852/1699] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:24:46 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 11:24:47.067906 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbedc32-4c7e-4f60-b96f-e717b20bc950 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:24:47.100283 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbedc32-4c7e-4f60-b96f-e717b20bc950 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:24:47.115448 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d179d4-cdf9-4b48-954c-2aa28fd71c9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:24:47.118773 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d179d4-cdf9-4b48-954c-2aa28fd71c9d admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:47.119180 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d179d4-cdf9-4b48-954c-2aa28fd71c9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:47.119478 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d179d4-cdf9-4b48-954c-2aa28fd71c9d admin admin] PATH_INFO: `/v3/projects/dd984f236ca24de1bfc1d53830385390/users/2db6348bebbc4ff084ecdb8494dbd8af/roles/e95c2ce63e114cc19a9dda6ad21e794f` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:47.154732 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d179d4-cdf9-4b48-954c-2aa28fd71c9d admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd984f236ca24de1bfc1d53830385390, user_id=2db6348bebbc4ff084ecdb8494dbd8af, role_id=e95c2ce63e114cc19a9dda6ad21e794f)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:24:47.157511 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d179d4-cdf9-4b48-954c-2aa28fd71c9d admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:24:47.176856 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5d6d76-c54b-43fe-867f-53720ffa38fe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:24:47.178490 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5d6d76-c54b-43fe-867f-53720ffa38fe service nova] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:24:47.232882 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 848/1700] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:24:46 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 710 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:24:47.276258 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 853/1701] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jun 29 11:24:47 2022] PUT /identity/v3/projects/dd984f236ca24de1bfc1d53830385390/users/2db6348bebbc4ff084ecdb8494dbd8af/roles/e95c2ce63e114cc19a9dda6ad21e794f => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:24:47.280177 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5d6d76-c54b-43fe-867f-53720ffa38fe service nova] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:24:47.309723 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5d6d76-c54b-43fe-867f-53720ffa38fe service nova] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:24:47.412455 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:24:47.437834 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc7e68b-bae8-41c0-b1c0-d04f1990f72a None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:47.438214 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc7e68b-bae8-41c0-b1c0-d04f1990f72a None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:47.438589 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc7e68b-bae8-41c0-b1c0-d04f1990f72a None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:47.440238 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 854/1702] 10.222.0.44 () {54 vars in 960 bytes} [Wed Jun 29 11:24:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 29 11:24:47.450149 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16290319-16c0-4519-9c22-5d6cf62ce21b None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:47.450383 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16290319-16c0-4519-9c22-5d6cf62ce21b None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:47.450815 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16290319-16c0-4519-9c22-5d6cf62ce21b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:47.503015 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-16290319-16c0-4519-9c22-5d6cf62ce21b None None] MFA Rules not processed for user `1a0505342b704d02977a5125957c2c9e`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:24:47.506298 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-16290319-16c0-4519-9c22-5d6cf62ce21b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:24:47.541535 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36522444-402d-451d-b5df-bfb6cd44f039 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:24:47.542413 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36522444-402d-451d-b5df-bfb6cd44f039 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:47.542786 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36522444-402d-451d-b5df-bfb6cd44f039 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:47.542953 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36522444-402d-451d-b5df-bfb6cd44f039 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:47.543695 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36522444-402d-451d-b5df-bfb6cd44f039 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:24:47.546616 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36522444-402d-451d-b5df-bfb6cd44f039 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:24:47.591877 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 849/1703] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:24:47 2022] GET /identity/v3/roles => generated 1420 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 11:24:47.597612 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36522444-402d-451d-b5df-bfb6cd44f039 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:24:47.621960 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36522444-402d-451d-b5df-bfb6cd44f039 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:24:47.641446 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f5f24f-e0fe-4e89-91a6-627aae983a02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:24:47.647886 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f5f24f-e0fe-4e89-91a6-627aae983a02 admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:47.648175 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f5f24f-e0fe-4e89-91a6-627aae983a02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:47.651204 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f5f24f-e0fe-4e89-91a6-627aae983a02 admin admin] PATH_INFO: `/v3/projects/dd984f236ca24de1bfc1d53830385390/users/2db6348bebbc4ff084ecdb8494dbd8af/roles/e95c2ce63e114cc19a9dda6ad21e794f` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:47.658440 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f5f24f-e0fe-4e89-91a6-627aae983a02 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd984f236ca24de1bfc1d53830385390, user_id=2db6348bebbc4ff084ecdb8494dbd8af, role_id=e95c2ce63e114cc19a9dda6ad21e794f)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:24:47.661528 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f5f24f-e0fe-4e89-91a6-627aae983a02 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:24:47.668352 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 855/1704] 10.222.0.44 () {58 vars in 1057 bytes} [Wed Jun 29 11:24:47 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 11:24:47.673927 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:24:47.677943 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49b1a5a-e46d-4b0c-84ce-801339139ce7 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:47.678236 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49b1a5a-e46d-4b0c-84ce-801339139ce7 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:47.678519 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49b1a5a-e46d-4b0c-84ce-801339139ce7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:47.686229 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 850/1705] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jun 29 11:24:47 2022] PUT /identity/v3/projects/dd984f236ca24de1bfc1d53830385390/users/2db6348bebbc4ff084ecdb8494dbd8af/roles/e95c2ce63e114cc19a9dda6ad21e794f => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:24:47.689825 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f5f24f-e0fe-4e89-91a6-627aae983a02 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:24:47.751202 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-c49b1a5a-e46d-4b0c-84ce-801339139ce7 None None] MFA Rules not processed for user `8effcd6f6be64f14b3c8bc71f8866a99`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:24:47.754426 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-c49b1a5a-e46d-4b0c-84ce-801339139ce7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:24:47.808303 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f5f24f-e0fe-4e89-91a6-627aae983a02 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:24:47.817437 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8f5a50-8cd3-4dbf-b291-416954e8d1df service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a0505342b704d02977a5125957c2c9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:24:47.818720 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8f5a50-8cd3-4dbf-b291-416954e8d1df service placement] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:47.819018 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8f5a50-8cd3-4dbf-b291-416954e8d1df service placement] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:47.819304 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8f5a50-8cd3-4dbf-b291-416954e8d1df service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:47.822767 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-1c8f5a50-8cd3-4dbf-b291-416954e8d1df service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:24:47.859405 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8f5a50-8cd3-4dbf-b291-416954e8d1df service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:24:47.861128 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8f5a50-8cd3-4dbf-b291-416954e8d1df service placement] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:24:47.901133 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 851/1706] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:24:47 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:24:47.902086 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 856/1707] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:24:47 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:24:47.903743 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933d0485-85a2-4bd7-ab90-ac556d69e531 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:47.903743 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933d0485-85a2-4bd7-ab90-ac556d69e531 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:47.903743 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933d0485-85a2-4bd7-ab90-ac556d69e531 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:47.921454 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a72676-f66d-41b4-80c5-55fd2d86023e None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:47.924736 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a72676-f66d-41b4-80c5-55fd2d86023e None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:47.925581 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a72676-f66d-41b4-80c5-55fd2d86023e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:47.962839 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-50a72676-f66d-41b4-80c5-55fd2d86023e None None] MFA Rules not processed for user `8effcd6f6be64f14b3c8bc71f8866a99`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:24:47.964403 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-50a72676-f66d-41b4-80c5-55fd2d86023e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:24:47.968980 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-933d0485-85a2-4bd7-ab90-ac556d69e531 None None] MFA Rules not processed for user `2db6348bebbc4ff084ecdb8494dbd8af`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:24:47.975491 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-933d0485-85a2-4bd7-ab90-ac556d69e531 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:24:47.979925 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 857/1708] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:24:47 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:24:47.992864 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a72676-f66d-41b4-80c5-55fd2d86023e None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:24:47.994579 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-50a72676-f66d-41b4-80c5-55fd2d86023e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:24:48.136068 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 852/1709] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:24:47 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:24:48.146824 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933d0485-85a2-4bd7-ab90-ac556d69e531 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:24:48.160412 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:24:48.198000 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a72676-f66d-41b4-80c5-55fd2d86023e None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:24:48.206492 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95177a61-df1f-44ff-889f-fb9a48c147bb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '796aeeff16df4739bddec1f6dee61bfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:24:48.207305 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95177a61-df1f-44ff-889f-fb9a48c147bb service nova] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:48.207595 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95177a61-df1f-44ff-889f-fb9a48c147bb service nova] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:48.210159 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95177a61-df1f-44ff-889f-fb9a48c147bb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:48.239245 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95177a61-df1f-44ff-889f-fb9a48c147bb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:24:48.241011 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95177a61-df1f-44ff-889f-fb9a48c147bb service nova] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:24:48.278254 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 858/1710] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:24:47 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 283 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:24:48.294200 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933d0485-85a2-4bd7-ab90-ac556d69e531 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:24:48.303580 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863c813f-d810-4569-b852-b9c54b70ac82 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff7931b75a924394b1af733febce0834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:24:48.304479 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863c813f-d810-4569-b852-b9c54b70ac82 service cinder] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:48.305359 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863c813f-d810-4569-b852-b9c54b70ac82 service cinder] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:48.305708 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863c813f-d810-4569-b852-b9c54b70ac82 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:48.323351 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863c813f-d810-4569-b852-b9c54b70ac82 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:24:48.325594 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863c813f-d810-4569-b852-b9c54b70ac82 service cinder] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:24:48.341871 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 853/1711] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:24:48 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:24:48.383170 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:24:48.415867 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95177a61-df1f-44ff-889f-fb9a48c147bb service nova] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:24:48.425174 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95177a61-df1f-44ff-889f-fb9a48c147bb service nova] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:24:48.432922 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf1c70d-23e9-44a9-82c9-9963b000b8ab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff7931b75a924394b1af733febce0834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:24:48.432922 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf1c70d-23e9-44a9-82c9-9963b000b8ab service cinder] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:48.433331 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf1c70d-23e9-44a9-82c9-9963b000b8ab service cinder] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:48.435926 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf1c70d-23e9-44a9-82c9-9963b000b8ab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:48.576691 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf1c70d-23e9-44a9-82c9-9963b000b8ab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:24:48.579434 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf1c70d-23e9-44a9-82c9-9963b000b8ab service cinder] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:24:48.592264 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 859/1712] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:24:48 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:24:48.815940 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:24:48.933990 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:24:49.516920 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e3cbce-07e7-46d1-a486-782cf517eba0 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:49.517909 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e3cbce-07e7-46d1-a486-782cf517eba0 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:49.518352 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e3cbce-07e7-46d1-a486-782cf517eba0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:49.550155 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-c5e3cbce-07e7-46d1-a486-782cf517eba0 None None] MFA Rules not processed for user `8effcd6f6be64f14b3c8bc71f8866a99`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:24:49.557943 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-c5e3cbce-07e7-46d1-a486-782cf517eba0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:24:49.582868 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 854/1713] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:24:49 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:24:51.673829 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b8555c-94d0-4eac-a729-d90fced89ea9 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:51.675583 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b8555c-94d0-4eac-a729-d90fced89ea9 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:51.675893 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b8555c-94d0-4eac-a729-d90fced89ea9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:51.729772 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-95b8555c-94d0-4eac-a729-d90fced89ea9 None None] MFA Rules not processed for user `f75e0b3528b14856b73dc441c52d1d73`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:24:51.732592 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-95b8555c-94d0-4eac-a729-d90fced89ea9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:24:51.762316 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:24:51.849016 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 860/1714] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:24:51 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:24:54.620823 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef760c7d-0bed-47c5-8431-15d2ca9bade0 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:54.621669 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef760c7d-0bed-47c5-8431-15d2ca9bade0 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:54.622091 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef760c7d-0bed-47c5-8431-15d2ca9bade0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:54.705103 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-ef760c7d-0bed-47c5-8431-15d2ca9bade0 None None] MFA Rules not processed for user `55487c2107a44c15b656b4865ee0d4fa`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:24:54.706369 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-ef760c7d-0bed-47c5-8431-15d2ca9bade0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:24:54.864344 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 855/1715] 10.222.0.44 () {58 vars in 1009 bytes} [Wed Jun 29 11:24:54 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:24:55.440042 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b8555c-94d0-4eac-a729-d90fced89ea9 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:24:55.442822 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-95b8555c-94d0-4eac-a729-d90fced89ea9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:24:55.588757 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b8555c-94d0-4eac-a729-d90fced89ea9 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:24:55.594520 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43c5e9d-5abd-4dd2-b93e-7fd66e78f5cf service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3b38493ba904759badc800e396befda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:24:55.595189 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43c5e9d-5abd-4dd2-b93e-7fd66e78f5cf service swift] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:55.595478 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43c5e9d-5abd-4dd2-b93e-7fd66e78f5cf service swift] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:55.595768 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43c5e9d-5abd-4dd2-b93e-7fd66e78f5cf service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:55.605351 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43c5e9d-5abd-4dd2-b93e-7fd66e78f5cf service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:24:55.607567 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43c5e9d-5abd-4dd2-b93e-7fd66e78f5cf service swift] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:24:55.617843 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 861/1716] 10.222.0.44 () {60 vars in 1384 bytes} [Wed Jun 29 11:24:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 29 11:24:56.501732 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c432ea-1436-49ec-b2fb-7d38e03640a2 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:24:56.502656 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c432ea-1436-49ec-b2fb-7d38e03640a2 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:24:56.503082 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c432ea-1436-49ec-b2fb-7d38e03640a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:24:56.540252 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-56c432ea-1436-49ec-b2fb-7d38e03640a2 None None] MFA Rules not processed for user `b5f4bb9f989e4024a5da28dced646b92`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:24:56.541474 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-56c432ea-1436-49ec-b2fb-7d38e03640a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:24:56.657746 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:24:56.741213 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 856/1717] 10.222.0.46 () {58 vars in 1015 bytes} [Wed Jun 29 11:24:56 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:25:05.180110 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43c5e9d-5abd-4dd2-b93e-7fd66e78f5cf service swift] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:05.182642 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-a43c5e9d-5abd-4dd2-b93e-7fd66e78f5cf service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:25:05.324069 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43c5e9d-5abd-4dd2-b93e-7fd66e78f5cf service swift] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:05.324069 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739f1962-3b45-465a-85ba-248cd8ae2610 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:05.324069 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739f1962-3b45-465a-85ba-248cd8ae2610 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:05.324069 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739f1962-3b45-465a-85ba-248cd8ae2610 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:05.324634 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739f1962-3b45-465a-85ba-248cd8ae2610 admin admin] PATH_INFO: `/v3/users/5a8f922dfd1f464f9b50a83521c610f1` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:05.324634 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739f1962-3b45-465a-85ba-248cd8ae2610 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a8f922dfd1f464f9b50a83521c610f1)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:05.324634 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739f1962-3b45-465a-85ba-248cd8ae2610 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:05.667073 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-739f1962-3b45-465a-85ba-248cd8ae2610 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a8f922dfd1f464f9b50a83521c610f1'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:05.668215 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-739f1962-3b45-465a-85ba-248cd8ae2610 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a8f922dfd1f464f9b50a83521c610f1'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:05.820029 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c432ea-1436-49ec-b2fb-7d38e03640a2 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:05.834198 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c432ea-1436-49ec-b2fb-7d38e03640a2 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:05.839500 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efae3d5-1ae3-4475-a2d8-3b90b6998c53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:05.841014 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efae3d5-1ae3-4475-a2d8-3b90b6998c53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:05.841303 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efae3d5-1ae3-4475-a2d8-3b90b6998c53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:05.841584 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efae3d5-1ae3-4475-a2d8-3b90b6998c53 admin admin] PATH_INFO: `/v3/users/1e4c45395e3445c583b7c8ce55c61d83` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:05.845053 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efae3d5-1ae3-4475-a2d8-3b90b6998c53 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e4c45395e3445c583b7c8ce55c61d83)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:05.847723 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efae3d5-1ae3-4475-a2d8-3b90b6998c53 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:05.897017 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-739f1962-3b45-465a-85ba-248cd8ae2610 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a8f922dfd1f464f9b50a83521c610f1'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:05.902137 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:25:05.906285 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-739f1962-3b45-465a-85ba-248cd8ae2610 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a8f922dfd1f464f9b50a83521c610f1'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:05.906285 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-739f1962-3b45-465a-85ba-248cd8ae2610 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a8f922dfd1f464f9b50a83521c610f1'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:05.942555 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 862/1718] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jun 29 11:25:05 2022] DELETE /identity/v3/users/5a8f922dfd1f464f9b50a83521c610f1 => generated 0 bytes in 763 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:25:05.946979 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b735be-4935-450d-925c-7c2b50a78971 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:05.947438 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b735be-4935-450d-925c-7c2b50a78971 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:05.947902 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b735be-4935-450d-925c-7c2b50a78971 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:05.975691 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-70b735be-4935-450d-925c-7c2b50a78971 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:25:05.977797 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-70b735be-4935-450d-925c-7c2b50a78971 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:25:06.053146 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 863/1719] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:25:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:25:06.064008 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b735be-4935-450d-925c-7c2b50a78971 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:06.064486 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-70b735be-4935-450d-925c-7c2b50a78971 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:25:06.124982 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b735be-4935-450d-925c-7c2b50a78971 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:06.136082 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26c6b3d-328d-4dd0-b99a-057499ca2eff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f4bb9f989e4024a5da28dced646b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:06.136769 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26c6b3d-328d-4dd0-b99a-057499ca2eff service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:06.137209 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26c6b3d-328d-4dd0-b99a-057499ca2eff service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:06.137592 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26c6b3d-328d-4dd0-b99a-057499ca2eff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:06.160691 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26c6b3d-328d-4dd0-b99a-057499ca2eff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:06.162347 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26c6b3d-328d-4dd0-b99a-057499ca2eff service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:06.168392 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-7efae3d5-1ae3-4475-a2d8-3b90b6998c53 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e4c45395e3445c583b7c8ce55c61d83'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:06.176404 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 864/1720] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:25:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 11:25:06.270329 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-7efae3d5-1ae3-4475-a2d8-3b90b6998c53 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e4c45395e3445c583b7c8ce55c61d83'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:06.271261 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-7efae3d5-1ae3-4475-a2d8-3b90b6998c53 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e4c45395e3445c583b7c8ce55c61d83'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:06.306697 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-7efae3d5-1ae3-4475-a2d8-3b90b6998c53 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e4c45395e3445c583b7c8ce55c61d83'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:06.307459 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-7efae3d5-1ae3-4475-a2d8-3b90b6998c53 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e4c45395e3445c583b7c8ce55c61d83'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:06.350701 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 857/1721] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jun 29 11:25:05 2022] DELETE /identity/v3/users/1e4c45395e3445c583b7c8ce55c61d83 => generated 0 bytes in 527 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:25:06.357196 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122ae1b3-731b-4ba7-922f-22e9727bfa6f None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:06.357762 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122ae1b3-731b-4ba7-922f-22e9727bfa6f None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:06.358353 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122ae1b3-731b-4ba7-922f-22e9727bfa6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:06.380671 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-122ae1b3-731b-4ba7-922f-22e9727bfa6f None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:25:06.382825 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-122ae1b3-731b-4ba7-922f-22e9727bfa6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:25:06.473803 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 865/1722] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:25:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:25:06.496711 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efae3d5-1ae3-4475-a2d8-3b90b6998c53 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:06.517988 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-7efae3d5-1ae3-4475-a2d8-3b90b6998c53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:25:06.573748 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efae3d5-1ae3-4475-a2d8-3b90b6998c53 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:06.596450 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6ec213-dde5-4638-baa1-109efeaabbbc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f4bb9f989e4024a5da28dced646b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:06.597120 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6ec213-dde5-4638-baa1-109efeaabbbc service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:06.597406 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6ec213-dde5-4638-baa1-109efeaabbbc service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:06.597684 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6ec213-dde5-4638-baa1-109efeaabbbc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:06.608796 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6ec213-dde5-4638-baa1-109efeaabbbc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:06.610733 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6ec213-dde5-4638-baa1-109efeaabbbc service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:06.640765 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 858/1723] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:25:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:25:06.910254 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122ae1b3-731b-4ba7-922f-22e9727bfa6f None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:06.926614 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-122ae1b3-731b-4ba7-922f-22e9727bfa6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:25:07.064230 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122ae1b3-731b-4ba7-922f-22e9727bfa6f None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:07.064230 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8184bbae-7d28-4e3e-922e-6a5da5251a65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:07.064230 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8184bbae-7d28-4e3e-922e-6a5da5251a65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:07.064230 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8184bbae-7d28-4e3e-922e-6a5da5251a65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:07.065361 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8184bbae-7d28-4e3e-922e-6a5da5251a65 admin admin] PATH_INFO: `/v3/projects/bd0fad0a12a944dea5f22527241a6471` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:07.065361 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8184bbae-7d28-4e3e-922e-6a5da5251a65 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd0fad0a12a944dea5f22527241a6471)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:07.065361 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8184bbae-7d28-4e3e-922e-6a5da5251a65 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:07.316179 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-8184bbae-7d28-4e3e-922e-6a5da5251a65 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd0fad0a12a944dea5f22527241a6471', 'request_id': 'req-8184bbae-7d28-4e3e-922e-6a5da5251a65'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:07.324615 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-8184bbae-7d28-4e3e-922e-6a5da5251a65 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd0fad0a12a944dea5f22527241a6471', 'request_id': 'req-8184bbae-7d28-4e3e-922e-6a5da5251a65'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:07.366346 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-8184bbae-7d28-4e3e-922e-6a5da5251a65 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd0fad0a12a944dea5f22527241a6471', 'request_id': 'req-8184bbae-7d28-4e3e-922e-6a5da5251a65'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:07.517210 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-8184bbae-7d28-4e3e-922e-6a5da5251a65 admin admin] The token cache is being invalidate because project bd0fad0a12a944dea5f22527241a6471 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 11:25:07.517210 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-8184bbae-7d28-4e3e-922e-6a5da5251a65 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:07.517823 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-8184bbae-7d28-4e3e-922e-6a5da5251a65 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:07.520942 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 866/1724] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jun 29 11:25:06 2022] DELETE /identity/v3/projects/bd0fad0a12a944dea5f22527241a6471 => generated 0 bytes in 612 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 11:25:07.526587 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb28f44-d499-4af9-a5e0-8a1dab5472a0 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:07.527097 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb28f44-d499-4af9-a5e0-8a1dab5472a0 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:07.527179 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb28f44-d499-4af9-a5e0-8a1dab5472a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:07.551163 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-bfb28f44-d499-4af9-a5e0-8a1dab5472a0 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:25:07.553127 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-bfb28f44-d499-4af9-a5e0-8a1dab5472a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:25:07.643130 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 859/1725] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:25:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:25:07.644299 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6695293-f41c-43a5-9f6a-8cda7bf86622 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:07.644508 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6695293-f41c-43a5-9f6a-8cda7bf86622 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:07.644807 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6695293-f41c-43a5-9f6a-8cda7bf86622 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:07.669256 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-e6695293-f41c-43a5-9f6a-8cda7bf86622 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:25:07.670676 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-e6695293-f41c-43a5-9f6a-8cda7bf86622 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:25:07.679651 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 867/1726] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:25:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:25:07.683940 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb28f44-d499-4af9-a5e0-8a1dab5472a0 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:07.694127 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:25:07.697194 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb28f44-d499-4af9-a5e0-8a1dab5472a0 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:07.704684 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6695293-f41c-43a5-9f6a-8cda7bf86622 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:07.706210 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd79e07-1e2c-4ba7-8195-8103f84503fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:07.706558 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-e6695293-f41c-43a5-9f6a-8cda7bf86622 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:25:07.707935 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd79e07-1e2c-4ba7-8195-8103f84503fd admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:07.708682 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd79e07-1e2c-4ba7-8195-8103f84503fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:07.709105 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd79e07-1e2c-4ba7-8195-8103f84503fd admin admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:07.710050 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd79e07-1e2c-4ba7-8195-8103f84503fd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:25:07.712034 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd79e07-1e2c-4ba7-8195-8103f84503fd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:07.714081 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd79e07-1e2c-4ba7-8195-8103f84503fd admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:07.724288 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6695293-f41c-43a5-9f6a-8cda7bf86622 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:07.733166 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 860/1727] 10.222.0.44 () {58 vars in 1152 bytes} [Wed Jun 29 11:25:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:25:07.736091 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32395c2-b35e-4aa0-ab4a-527f62c34495 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:07.737010 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32395c2-b35e-4aa0-ab4a-527f62c34495 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:07.737294 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32395c2-b35e-4aa0-ab4a-527f62c34495 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:07.737568 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32395c2-b35e-4aa0-ab4a-527f62c34495 admin admin] PATH_INFO: `/v3/projects/e2190591a56948758833f8895fbdc3b8` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:07.738838 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32395c2-b35e-4aa0-ab4a-527f62c34495 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e2190591a56948758833f8895fbdc3b8)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:07.740835 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd79e07-1e2c-4ba7-8195-8103f84503fd admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:07.740969 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32395c2-b35e-4aa0-ab4a-527f62c34495 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:07.756734 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd79e07-1e2c-4ba7-8195-8103f84503fd admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:07.760696 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107f75b8-9a11-42f0-b5ae-8e0fc3566b2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:07.761470 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107f75b8-9a11-42f0-b5ae-8e0fc3566b2d admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:07.762105 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107f75b8-9a11-42f0-b5ae-8e0fc3566b2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:07.762734 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107f75b8-9a11-42f0-b5ae-8e0fc3566b2d admin admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:07.763569 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107f75b8-9a11-42f0-b5ae-8e0fc3566b2d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:07.765622 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107f75b8-9a11-42f0-b5ae-8e0fc3566b2d admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:07.879995 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-c32395c2-b35e-4aa0-ab4a-527f62c34495 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e2190591a56948758833f8895fbdc3b8', 'request_id': 'req-c32395c2-b35e-4aa0-ab4a-527f62c34495'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:07.886658 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-c32395c2-b35e-4aa0-ab4a-527f62c34495 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e2190591a56948758833f8895fbdc3b8', 'request_id': 'req-c32395c2-b35e-4aa0-ab4a-527f62c34495'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:07.892928 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-c32395c2-b35e-4aa0-ab4a-527f62c34495 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e2190591a56948758833f8895fbdc3b8', 'request_id': 'req-c32395c2-b35e-4aa0-ab4a-527f62c34495'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:07.957745 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-c32395c2-b35e-4aa0-ab4a-527f62c34495 admin admin] The token cache is being invalidate because project e2190591a56948758833f8895fbdc3b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 11:25:07.958091 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-c32395c2-b35e-4aa0-ab4a-527f62c34495 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:07.958749 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-c32395c2-b35e-4aa0-ab4a-527f62c34495 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:07.963724 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 861/1728] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jun 29 11:25:07 2022] POST /identity/v3/projects => generated 365 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:25:07.964037 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 868/1729] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jun 29 11:25:07 2022] DELETE /identity/v3/projects/e2190591a56948758833f8895fbdc3b8 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:25:07.964697 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32395c2-b35e-4aa0-ab4a-527f62c34495 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:07.964804 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107f75b8-9a11-42f0-b5ae-8e0fc3566b2d admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:07.966626 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-c32395c2-b35e-4aa0-ab4a-527f62c34495 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:25:07.966746 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-107f75b8-9a11-42f0-b5ae-8e0fc3566b2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:25:08.030746 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107f75b8-9a11-42f0-b5ae-8e0fc3566b2d admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:08.033858 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32395c2-b35e-4aa0-ab4a-527f62c34495 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:08.035403 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645f9dae-db51-4ebb-a2c5-2b81d62dcf3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:08.036108 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645f9dae-db51-4ebb-a2c5-2b81d62dcf3d admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:08.036517 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645f9dae-db51-4ebb-a2c5-2b81d62dcf3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:08.036726 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645f9dae-db51-4ebb-a2c5-2b81d62dcf3d admin admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:08.037414 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645f9dae-db51-4ebb-a2c5-2b81d62dcf3d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:08.039460 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645f9dae-db51-4ebb-a2c5-2b81d62dcf3d admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:08.039596 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:25:08.040430 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0777133d-9852-4a13-bebc-d25823df7105 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:08.041354 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0777133d-9852-4a13-bebc-d25823df7105 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:08.041638 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0777133d-9852-4a13-bebc-d25823df7105 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:08.041914 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0777133d-9852-4a13-bebc-d25823df7105 admin admin] PATH_INFO: `/v3/users/2db6348bebbc4ff084ecdb8494dbd8af` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:08.043114 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0777133d-9852-4a13-bebc-d25823df7105 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2db6348bebbc4ff084ecdb8494dbd8af)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:08.045032 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0777133d-9852-4a13-bebc-d25823df7105 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:08.157119 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 862/1730] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jun 29 11:25:07 2022] POST /identity/v3/users => generated 283 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:25:08.161126 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645f9dae-db51-4ebb-a2c5-2b81d62dcf3d admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:08.170853 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645f9dae-db51-4ebb-a2c5-2b81d62dcf3d admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:08.175439 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639b5750-d3db-4fe2-b07f-8a05ce22ea00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:08.176407 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639b5750-d3db-4fe2-b07f-8a05ce22ea00 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:08.176743 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639b5750-d3db-4fe2-b07f-8a05ce22ea00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:08.176965 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639b5750-d3db-4fe2-b07f-8a05ce22ea00 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:08.177693 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639b5750-d3db-4fe2-b07f-8a05ce22ea00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:08.180043 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639b5750-d3db-4fe2-b07f-8a05ce22ea00 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:08.189533 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 863/1731] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:25:08 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 11:25:08.194029 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639b5750-d3db-4fe2-b07f-8a05ce22ea00 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:08.204924 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639b5750-d3db-4fe2-b07f-8a05ce22ea00 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:08.209820 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68386dbc-3717-416a-8241-1b7a6f6f0446 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:08.215358 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68386dbc-3717-416a-8241-1b7a6f6f0446 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:08.215358 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68386dbc-3717-416a-8241-1b7a6f6f0446 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:08.215358 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68386dbc-3717-416a-8241-1b7a6f6f0446 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:08.215358 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68386dbc-3717-416a-8241-1b7a6f6f0446 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:08.215358 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68386dbc-3717-416a-8241-1b7a6f6f0446 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:08.226029 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 864/1732] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:25:08 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 11:25:08.230028 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68386dbc-3717-416a-8241-1b7a6f6f0446 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:08.240024 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68386dbc-3717-416a-8241-1b7a6f6f0446 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:08.246516 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a293e80-088f-4d93-8998-4aabfb9adb69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:08.248814 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a293e80-088f-4d93-8998-4aabfb9adb69 admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:08.248814 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a293e80-088f-4d93-8998-4aabfb9adb69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:08.248814 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a293e80-088f-4d93-8998-4aabfb9adb69 admin admin] PATH_INFO: `/v3/projects/fd6481343c5c420a9d3ced7e316b7ca7/users/4f2f25a7608d403ea74525a7270c28a4/roles/e1c88eebe911432cab6a07fbeb84653b` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:08.263155 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a293e80-088f-4d93-8998-4aabfb9adb69 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd6481343c5c420a9d3ced7e316b7ca7, user_id=4f2f25a7608d403ea74525a7270c28a4, role_id=e1c88eebe911432cab6a07fbeb84653b)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:08.265840 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a293e80-088f-4d93-8998-4aabfb9adb69 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:08.282494 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-0777133d-9852-4a13-bebc-d25823df7105 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2db6348bebbc4ff084ecdb8494dbd8af'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:08.283135 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-0777133d-9852-4a13-bebc-d25823df7105 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2db6348bebbc4ff084ecdb8494dbd8af'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:08.348701 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-0777133d-9852-4a13-bebc-d25823df7105 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2db6348bebbc4ff084ecdb8494dbd8af'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:08.356233 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-0777133d-9852-4a13-bebc-d25823df7105 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2db6348bebbc4ff084ecdb8494dbd8af'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:08.356891 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-0777133d-9852-4a13-bebc-d25823df7105 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2db6348bebbc4ff084ecdb8494dbd8af'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:08.357719 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 865/1733] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jun 29 11:25:08 2022] PUT /identity/v3/projects/fd6481343c5c420a9d3ced7e316b7ca7/users/4f2f25a7608d403ea74525a7270c28a4/roles/e1c88eebe911432cab6a07fbeb84653b => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 11:25:08.369407 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773d0bc8-c363-4365-92b6-fdb15bd4549e None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:08.369749 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773d0bc8-c363-4365-92b6-fdb15bd4549e None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:08.370070 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773d0bc8-c363-4365-92b6-fdb15bd4549e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:08.375718 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 869/1734] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jun 29 11:25:07 2022] DELETE /identity/v3/users/2db6348bebbc4ff084ecdb8494dbd8af => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:25:08.396383 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-773d0bc8-c363-4365-92b6-fdb15bd4549e None None] MFA Rules not processed for user `4f2f25a7608d403ea74525a7270c28a4`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:25:08.397609 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-773d0bc8-c363-4365-92b6-fdb15bd4549e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:25:08.477522 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 866/1735] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:25:08 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:25:08.481642 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0777133d-9852-4a13-bebc-d25823df7105 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:08.485761 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-0777133d-9852-4a13-bebc-d25823df7105 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:25:08.594455 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0777133d-9852-4a13-bebc-d25823df7105 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:08.599679 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2fb517-3602-401d-8dee-957c88ba9fdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:08.600670 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2fb517-3602-401d-8dee-957c88ba9fdf admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:08.601479 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2fb517-3602-401d-8dee-957c88ba9fdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:08.601759 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2fb517-3602-401d-8dee-957c88ba9fdf admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:08.602452 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2fb517-3602-401d-8dee-957c88ba9fdf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:08.604493 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2fb517-3602-401d-8dee-957c88ba9fdf admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:08.709957 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 870/1736] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jun 29 11:25:08 2022] POST /identity/v3/projects => generated 363 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:25:08.713997 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773d0bc8-c363-4365-92b6-fdb15bd4549e None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:08.727336 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773d0bc8-c363-4365-92b6-fdb15bd4549e None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:08.796523 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61de9b13-d6ce-42b8-b9ef-8cea7c7cc2f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:08.797189 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61de9b13-d6ce-42b8-b9ef-8cea7c7cc2f2 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:08.797479 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61de9b13-d6ce-42b8-b9ef-8cea7c7cc2f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:08.797751 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61de9b13-d6ce-42b8-b9ef-8cea7c7cc2f2 admin admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:08.798506 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61de9b13-d6ce-42b8-b9ef-8cea7c7cc2f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:08.800406 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61de9b13-d6ce-42b8-b9ef-8cea7c7cc2f2 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:08.889677 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2fb517-3602-401d-8dee-957c88ba9fdf admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:08.891451 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-fe2fb517-3602-401d-8dee-957c88ba9fdf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:25:08.896202 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 867/1737] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jun 29 11:25:08 2022] POST /identity/v3/users => generated 288 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 11:25:08.907123 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2fb517-3602-401d-8dee-957c88ba9fdf admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:08.914496 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61de9b13-d6ce-42b8-b9ef-8cea7c7cc2f2 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:08.914577 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88dcfb6d-7298-46b2-bc89-6590b16a5639 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:08.914674 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dcfb6d-7298-46b2-bc89-6590b16a5639 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:08.914918 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dcfb6d-7298-46b2-bc89-6590b16a5639 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:08.915194 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dcfb6d-7298-46b2-bc89-6590b16a5639 admin admin] PATH_INFO: `/v3/projects/dd984f236ca24de1bfc1d53830385390` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:08.918653 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88dcfb6d-7298-46b2-bc89-6590b16a5639 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd984f236ca24de1bfc1d53830385390)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:08.920647 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88dcfb6d-7298-46b2-bc89-6590b16a5639 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:08.925050 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61de9b13-d6ce-42b8-b9ef-8cea7c7cc2f2 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:08.930590 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4a77fd-1d2d-4660-8e98-1ddc819c5611 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:08.931413 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4a77fd-1d2d-4660-8e98-1ddc819c5611 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:08.931799 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4a77fd-1d2d-4660-8e98-1ddc819c5611 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:08.932195 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4a77fd-1d2d-4660-8e98-1ddc819c5611 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:08.933149 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4a77fd-1d2d-4660-8e98-1ddc819c5611 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:08.934895 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4a77fd-1d2d-4660-8e98-1ddc819c5611 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:08.946859 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 868/1738] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:25:08 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 11:25:08.950749 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4a77fd-1d2d-4660-8e98-1ddc819c5611 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:08.961589 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4a77fd-1d2d-4660-8e98-1ddc819c5611 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:08.966521 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35688a2-1491-455d-abb9-ef1745fd13da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:08.967467 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35688a2-1491-455d-abb9-ef1745fd13da admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:08.967783 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35688a2-1491-455d-abb9-ef1745fd13da admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:08.968093 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35688a2-1491-455d-abb9-ef1745fd13da admin admin] PATH_INFO: `/v3/projects/c5f1a2c361284f84a445b0991532d9ee/users/e7c5afe798434bbf8f6165152065482a/roles/e95c2ce63e114cc19a9dda6ad21e794f` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:08.984369 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35688a2-1491-455d-abb9-ef1745fd13da admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5f1a2c361284f84a445b0991532d9ee, user_id=e7c5afe798434bbf8f6165152065482a, role_id=e95c2ce63e114cc19a9dda6ad21e794f)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:08.987019 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35688a2-1491-455d-abb9-ef1745fd13da admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:09.081732 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 869/1739] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jun 29 11:25:08 2022] PUT /identity/v3/projects/c5f1a2c361284f84a445b0991532d9ee/users/e7c5afe798434bbf8f6165152065482a/roles/e95c2ce63e114cc19a9dda6ad21e794f => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 11:25:09.086026 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35688a2-1491-455d-abb9-ef1745fd13da admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:09.097399 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-88dcfb6d-7298-46b2-bc89-6590b16a5639 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd984f236ca24de1bfc1d53830385390', 'request_id': 'req-88dcfb6d-7298-46b2-bc89-6590b16a5639'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:09.105175 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-88dcfb6d-7298-46b2-bc89-6590b16a5639 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd984f236ca24de1bfc1d53830385390', 'request_id': 'req-88dcfb6d-7298-46b2-bc89-6590b16a5639'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:09.111058 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-88dcfb6d-7298-46b2-bc89-6590b16a5639 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd984f236ca24de1bfc1d53830385390', 'request_id': 'req-88dcfb6d-7298-46b2-bc89-6590b16a5639'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:09.154016 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:25:09.167148 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35688a2-1491-455d-abb9-ef1745fd13da admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:09.189825 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-88dcfb6d-7298-46b2-bc89-6590b16a5639 admin admin] The token cache is being invalidate because project dd984f236ca24de1bfc1d53830385390 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 11:25:09.189825 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-88dcfb6d-7298-46b2-bc89-6590b16a5639 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:09.189825 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-88dcfb6d-7298-46b2-bc89-6590b16a5639 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:09.190378 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 871/1740] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jun 29 11:25:08 2022] DELETE /identity/v3/projects/dd984f236ca24de1bfc1d53830385390 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:25:09.194724 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77528ed5-458e-476e-8b25-9a04cb04f358 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:09.197287 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77528ed5-458e-476e-8b25-9a04cb04f358 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:09.197748 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77528ed5-458e-476e-8b25-9a04cb04f358 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:09.217594 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-77528ed5-458e-476e-8b25-9a04cb04f358 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:25:09.219499 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-77528ed5-458e-476e-8b25-9a04cb04f358 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:25:09.231273 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9a3728-b43e-4076-8251-b455b6b10e11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:09.232026 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9a3728-b43e-4076-8251-b455b6b10e11 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:09.237630 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9a3728-b43e-4076-8251-b455b6b10e11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:09.237630 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9a3728-b43e-4076-8251-b455b6b10e11 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:09.237630 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9a3728-b43e-4076-8251-b455b6b10e11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:09.237630 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9a3728-b43e-4076-8251-b455b6b10e11 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:09.246128 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 870/1741] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:25:09 2022] GET /identity/v3/roles => generated 1420 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 11:25:09.250111 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9a3728-b43e-4076-8251-b455b6b10e11 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:09.252749 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-6a9a3728-b43e-4076-8251-b455b6b10e11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:25:09.270103 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9a3728-b43e-4076-8251-b455b6b10e11 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:09.285694 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84327af9-a4f5-4a1b-aade-37500c7be033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:09.286698 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84327af9-a4f5-4a1b-aade-37500c7be033 admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:09.287046 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84327af9-a4f5-4a1b-aade-37500c7be033 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:09.287396 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84327af9-a4f5-4a1b-aade-37500c7be033 admin admin] PATH_INFO: `/v3/projects/c5f1a2c361284f84a445b0991532d9ee/users/e7c5afe798434bbf8f6165152065482a/roles/e95c2ce63e114cc19a9dda6ad21e794f` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:09.290226 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84327af9-a4f5-4a1b-aade-37500c7be033 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5f1a2c361284f84a445b0991532d9ee, user_id=e7c5afe798434bbf8f6165152065482a, role_id=e95c2ce63e114cc19a9dda6ad21e794f)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:09.293187 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84327af9-a4f5-4a1b-aade-37500c7be033 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:09.304833 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 871/1742] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jun 29 11:25:09 2022] PUT /identity/v3/projects/c5f1a2c361284f84a445b0991532d9ee/users/e7c5afe798434bbf8f6165152065482a/roles/e95c2ce63e114cc19a9dda6ad21e794f => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:25:09.311912 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 872/1743] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:25:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 11:25:09.314845 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d38ad62-0277-4b9a-bbda-19027a744e27 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:09.315113 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d38ad62-0277-4b9a-bbda-19027a744e27 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:09.315391 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d38ad62-0277-4b9a-bbda-19027a744e27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:09.318971 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f614639-23d6-4cc6-91d5-35a2c73078fe None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:09.320404 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f614639-23d6-4cc6-91d5-35a2c73078fe None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:09.320404 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f614639-23d6-4cc6-91d5-35a2c73078fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:09.342172 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-0f614639-23d6-4cc6-91d5-35a2c73078fe None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:25:09.343596 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-0f614639-23d6-4cc6-91d5-35a2c73078fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:25:09.354864 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-2d38ad62-0277-4b9a-bbda-19027a744e27 None None] MFA Rules not processed for user `e7c5afe798434bbf8f6165152065482a`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:25:09.358811 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-2d38ad62-0277-4b9a-bbda-19027a744e27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:25:09.378570 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:25:09.452304 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 873/1744] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:25:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 11:25:09.454018 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f614639-23d6-4cc6-91d5-35a2c73078fe None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:09.455845 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-0f614639-23d6-4cc6-91d5-35a2c73078fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:25:09.473301 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 872/1745] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:25:09 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:25:09.474334 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d38ad62-0277-4b9a-bbda-19027a744e27 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:09.476497 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-2d38ad62-0277-4b9a-bbda-19027a744e27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:25:09.520578 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f614639-23d6-4cc6-91d5-35a2c73078fe None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:09.525619 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e15265-fcc6-4518-8923-1c6876381c8d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f4bb9f989e4024a5da28dced646b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:09.526291 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e15265-fcc6-4518-8923-1c6876381c8d service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:09.526716 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e15265-fcc6-4518-8923-1c6876381c8d service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:09.527002 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e15265-fcc6-4518-8923-1c6876381c8d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:09.529986 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-64e15265-fcc6-4518-8923-1c6876381c8d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:25:09.541377 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e15265-fcc6-4518-8923-1c6876381c8d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:09.542866 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e15265-fcc6-4518-8923-1c6876381c8d service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:09.551503 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 874/1746] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:25:09 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:25:09.552311 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e15265-fcc6-4518-8923-1c6876381c8d service neutron] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:09.554526 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-64e15265-fcc6-4518-8923-1c6876381c8d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:25:09.554826 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d38ad62-0277-4b9a-bbda-19027a744e27 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:09.559710 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ee91b2-d707-4093-ad15-a91003df9272 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f4bb9f989e4024a5da28dced646b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:09.566164 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ee91b2-d707-4093-ad15-a91003df9272 service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:09.566458 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ee91b2-d707-4093-ad15-a91003df9272 service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:09.566741 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ee91b2-d707-4093-ad15-a91003df9272 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:09.571868 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e15265-fcc6-4518-8923-1c6876381c8d service neutron] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:09.581992 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ecba06-58a5-4e92-9671-76035bbfa4bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f4bb9f989e4024a5da28dced646b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:09.582848 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ecba06-58a5-4e92-9671-76035bbfa4bc service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:09.583280 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ecba06-58a5-4e92-9671-76035bbfa4bc service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:09.583700 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ee91b2-d707-4093-ad15-a91003df9272 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:09.583880 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ecba06-58a5-4e92-9671-76035bbfa4bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:09.586097 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ee91b2-d707-4093-ad15-a91003df9272 service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:09.599549 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 873/1747] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:25:09 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:25:09.600431 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ee91b2-d707-4093-ad15-a91003df9272 service neutron] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:09.605749 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ecba06-58a5-4e92-9671-76035bbfa4bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:09.607553 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ecba06-58a5-4e92-9671-76035bbfa4bc service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:09.619023 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 875/1748] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:25:09 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:25:09.619023 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ecba06-58a5-4e92-9671-76035bbfa4bc service neutron] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:09.620020 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-e7ecba06-58a5-4e92-9671-76035bbfa4bc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:25:09.635079 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ee91b2-d707-4093-ad15-a91003df9272 service neutron] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:09.645138 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6aed711-2b51-4223-b941-30072f45c77e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:09.647514 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6aed711-2b51-4223-b941-30072f45c77e admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:09.647514 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6aed711-2b51-4223-b941-30072f45c77e admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:09.647514 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6aed711-2b51-4223-b941-30072f45c77e admin admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:09.647514 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6aed711-2b51-4223-b941-30072f45c77e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:25:09.647514 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6aed711-2b51-4223-b941-30072f45c77e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:09.649766 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6aed711-2b51-4223-b941-30072f45c77e admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:09.681067 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 874/1749] 10.222.0.44 () {58 vars in 1152 bytes} [Wed Jun 29 11:25:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:25:09.686856 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6aed711-2b51-4223-b941-30072f45c77e admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:09.714562 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6aed711-2b51-4223-b941-30072f45c77e admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:09.720425 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c456ba8a-26a8-42a6-907b-2b1d194c6e73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:09.721058 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c456ba8a-26a8-42a6-907b-2b1d194c6e73 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:09.721293 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c456ba8a-26a8-42a6-907b-2b1d194c6e73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:09.721527 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c456ba8a-26a8-42a6-907b-2b1d194c6e73 admin admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:09.722246 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c456ba8a-26a8-42a6-907b-2b1d194c6e73 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:09.727132 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c456ba8a-26a8-42a6-907b-2b1d194c6e73 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:09.753851 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ecba06-58a5-4e92-9671-76035bbfa4bc service neutron] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:09.767809 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ffd23d-bc95-419c-9886-4b755879ed85 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff7931b75a924394b1af733febce0834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:09.770109 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ffd23d-bc95-419c-9886-4b755879ed85 service cinder] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:09.770847 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ffd23d-bc95-419c-9886-4b755879ed85 service cinder] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:09.771371 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ffd23d-bc95-419c-9886-4b755879ed85 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:09.794152 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ffd23d-bc95-419c-9886-4b755879ed85 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:09.795714 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ffd23d-bc95-419c-9886-4b755879ed85 service cinder] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:09.807220 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 876/1750] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:25:09 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:25:09.892202 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 875/1751] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jun 29 11:25:09 2022] POST /identity/v3/projects => generated 399 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 11:25:09.894596 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ffd23d-bc95-419c-9886-4b755879ed85 service cinder] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:09.926518 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ffd23d-bc95-419c-9886-4b755879ed85 service cinder] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:10.010297 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c87ce9a-8e78-4aa6-a14b-506e008806cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:10.010998 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c87ce9a-8e78-4aa6-a14b-506e008806cc admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:10.011341 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c87ce9a-8e78-4aa6-a14b-506e008806cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:10.011550 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c87ce9a-8e78-4aa6-a14b-506e008806cc admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:10.012702 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c87ce9a-8e78-4aa6-a14b-506e008806cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:10.014888 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c87ce9a-8e78-4aa6-a14b-506e008806cc admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:10.093682 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 877/1752] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jun 29 11:25:09 2022] POST /identity/v3/users => generated 300 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:25:10.099256 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c456ba8a-26a8-42a6-907b-2b1d194c6e73 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:10.113041 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c456ba8a-26a8-42a6-907b-2b1d194c6e73 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:10.118859 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b99a34-c1bf-47f9-ae81-91bdf63c4920 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:10.119089 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b99a34-c1bf-47f9-ae81-91bdf63c4920 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:10.119381 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b99a34-c1bf-47f9-ae81-91bdf63c4920 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:10.119658 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b99a34-c1bf-47f9-ae81-91bdf63c4920 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:10.120436 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b99a34-c1bf-47f9-ae81-91bdf63c4920 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:10.135439 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b99a34-c1bf-47f9-ae81-91bdf63c4920 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:10.163168 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 876/1753] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:25:10 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 11:25:10.164021 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c87ce9a-8e78-4aa6-a14b-506e008806cc admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:10.176910 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c87ce9a-8e78-4aa6-a14b-506e008806cc admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:10.183288 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664c5cd3-ec78-4282-971d-d1da9c2f1c57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:10.184729 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664c5cd3-ec78-4282-971d-d1da9c2f1c57 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:10.185149 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664c5cd3-ec78-4282-971d-d1da9c2f1c57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:10.185445 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664c5cd3-ec78-4282-971d-d1da9c2f1c57 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:10.186190 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664c5cd3-ec78-4282-971d-d1da9c2f1c57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:10.188264 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664c5cd3-ec78-4282-971d-d1da9c2f1c57 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:10.203550 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 878/1754] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:25:10 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 11:25:10.205882 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b99a34-c1bf-47f9-ae81-91bdf63c4920 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:10.219601 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b99a34-c1bf-47f9-ae81-91bdf63c4920 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:10.228327 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e770bcf3-7a7c-493a-85f2-07a38dc128cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:10.228327 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e770bcf3-7a7c-493a-85f2-07a38dc128cf admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:10.228327 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e770bcf3-7a7c-493a-85f2-07a38dc128cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:10.228327 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e770bcf3-7a7c-493a-85f2-07a38dc128cf admin admin] PATH_INFO: `/v3/projects/42a37aad69cf49708fdb70896cb09ba1/users/ca20ad93e1394f129531756c7bbc2f98/roles/e1c88eebe911432cab6a07fbeb84653b` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:10.249435 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e770bcf3-7a7c-493a-85f2-07a38dc128cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=42a37aad69cf49708fdb70896cb09ba1, user_id=ca20ad93e1394f129531756c7bbc2f98, role_id=e1c88eebe911432cab6a07fbeb84653b)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:10.253923 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e770bcf3-7a7c-493a-85f2-07a38dc128cf admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:10.262064 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:25:10.367471 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 877/1755] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jun 29 11:25:10 2022] PUT /identity/v3/projects/42a37aad69cf49708fdb70896cb09ba1/users/ca20ad93e1394f129531756c7bbc2f98/roles/e1c88eebe911432cab6a07fbeb84653b => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:25:10.380670 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95ae992-9c72-458f-bf5d-88cde87013be None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:10.380963 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95ae992-9c72-458f-bf5d-88cde87013be None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:10.381245 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95ae992-9c72-458f-bf5d-88cde87013be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:10.419140 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-a95ae992-9c72-458f-bf5d-88cde87013be None None] MFA Rules not processed for user `ca20ad93e1394f129531756c7bbc2f98`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:25:10.420368 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-a95ae992-9c72-458f-bf5d-88cde87013be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:25:10.496977 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 879/1756] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:25:10 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 11:25:10.497879 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e770bcf3-7a7c-493a-85f2-07a38dc128cf admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:10.584787 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e770bcf3-7a7c-493a-85f2-07a38dc128cf admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:10.590054 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d89a0b2-32fd-48a8-b205-2084271de7b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:10.592493 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d89a0b2-32fd-48a8-b205-2084271de7b1 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:10.592493 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d89a0b2-32fd-48a8-b205-2084271de7b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:10.592493 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d89a0b2-32fd-48a8-b205-2084271de7b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:10.592493 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d89a0b2-32fd-48a8-b205-2084271de7b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:10.595377 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d89a0b2-32fd-48a8-b205-2084271de7b1 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:10.621369 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:DELETE=14,keystone:SELECT=83,keystone:INSERT=3 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:25:10.729013 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 878/1757] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jun 29 11:25:10 2022] POST /identity/v3/projects => generated 399 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:25:10.731510 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95ae992-9c72-458f-bf5d-88cde87013be None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:10.767351 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95ae992-9c72-458f-bf5d-88cde87013be None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:10.941771 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a49944-7478-41c0-91b5-168ff75b8aac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:10.944767 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a49944-7478-41c0-91b5-168ff75b8aac admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:10.945074 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a49944-7478-41c0-91b5-168ff75b8aac admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:10.945356 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a49944-7478-41c0-91b5-168ff75b8aac admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:10.948471 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a49944-7478-41c0-91b5-168ff75b8aac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:10.954643 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a49944-7478-41c0-91b5-168ff75b8aac admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:11.339780 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 880/1758] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jun 29 11:25:10 2022] POST /identity/v3/users => generated 306 bytes in 609 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:25:11.341172 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d89a0b2-32fd-48a8-b205-2084271de7b1 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:11.356890 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d89a0b2-32fd-48a8-b205-2084271de7b1 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:11.361634 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3839455-5996-4396-bebe-93c98d6ca2f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:11.362464 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3839455-5996-4396-bebe-93c98d6ca2f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:11.362745 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3839455-5996-4396-bebe-93c98d6ca2f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:11.363427 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3839455-5996-4396-bebe-93c98d6ca2f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:11.364478 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3839455-5996-4396-bebe-93c98d6ca2f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:11.366508 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3839455-5996-4396-bebe-93c98d6ca2f9 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:11.379798 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 879/1759] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:25:11 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 11:25:11.388606 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a49944-7478-41c0-91b5-168ff75b8aac admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:11.399345 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a49944-7478-41c0-91b5-168ff75b8aac admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:11.408169 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5348bf06-e7e1-416a-a4a4-b98e92244474 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:11.409732 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5348bf06-e7e1-416a-a4a4-b98e92244474 admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:11.410240 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5348bf06-e7e1-416a-a4a4-b98e92244474 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:11.411413 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5348bf06-e7e1-416a-a4a4-b98e92244474 admin admin] PATH_INFO: `/v3/projects/381814a417d7459c8a3fe9b92991c46f/users/4b960532407841e1886eae53db74cc96/roles/e95c2ce63e114cc19a9dda6ad21e794f` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:11.434961 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5348bf06-e7e1-416a-a4a4-b98e92244474 admin admin] RBAC: Authorizing `identity:create_grant(project_id=381814a417d7459c8a3fe9b92991c46f, user_id=4b960532407841e1886eae53db74cc96, role_id=e95c2ce63e114cc19a9dda6ad21e794f)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:11.438287 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5348bf06-e7e1-416a-a4a4-b98e92244474 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:11.530473 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 881/1760] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jun 29 11:25:11 2022] PUT /identity/v3/projects/381814a417d7459c8a3fe9b92991c46f/users/4b960532407841e1886eae53db74cc96/roles/e95c2ce63e114cc19a9dda6ad21e794f => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:25:11.540247 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3839455-5996-4396-bebe-93c98d6ca2f9 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:11.550664 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3839455-5996-4396-bebe-93c98d6ca2f9 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:11.688661 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b71779-1b68-4b6c-bcba-8270c1f33c95 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:11.689210 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b71779-1b68-4b6c-bcba-8270c1f33c95 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:11.689665 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b71779-1b68-4b6c-bcba-8270c1f33c95 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:11.691709 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 882/1761] 10.222.0.44 () {54 vars in 921 bytes} [Wed Jun 29 11:25:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 29 11:25:11.713916 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b71779-1b68-4b6c-bcba-8270c1f33c95 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:11.714005 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb8699b-4ee0-4e75-8dc8-85d7cbc28632 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:11.714600 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb8699b-4ee0-4e75-8dc8-85d7cbc28632 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:11.714891 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb8699b-4ee0-4e75-8dc8-85d7cbc28632 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:11.715177 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb8699b-4ee0-4e75-8dc8-85d7cbc28632 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:11.717646 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb8699b-4ee0-4e75-8dc8-85d7cbc28632 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:11.721495 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb8699b-4ee0-4e75-8dc8-85d7cbc28632 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:11.755551 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 880/1762] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:25:11 2022] GET /identity/v3/roles => generated 1420 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 11:25:11.756487 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b71779-1b68-4b6c-bcba-8270c1f33c95 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:11.759080 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb8699b-4ee0-4e75-8dc8-85d7cbc28632 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:11.768020 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7bbb8d-64fd-43a8-855a-07fb72e889f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:11.770035 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7bbb8d-64fd-43a8-855a-07fb72e889f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:11.771812 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7bbb8d-64fd-43a8-855a-07fb72e889f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:11.772377 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7bbb8d-64fd-43a8-855a-07fb72e889f1 admin admin] PATH_INFO: `/v3/users/4f2f25a7608d403ea74525a7270c28a4` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:11.776450 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7bbb8d-64fd-43a8-855a-07fb72e889f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f2f25a7608d403ea74525a7270c28a4)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:11.780528 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7bbb8d-64fd-43a8-855a-07fb72e889f1 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:11.793385 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb8699b-4ee0-4e75-8dc8-85d7cbc28632 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:11.813332 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5268825-cb37-4102-9f07-7536f4ccce88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:11.815996 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5268825-cb37-4102-9f07-7536f4ccce88 admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:11.818343 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5268825-cb37-4102-9f07-7536f4ccce88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:11.821533 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5268825-cb37-4102-9f07-7536f4ccce88 admin admin] PATH_INFO: `/v3/projects/381814a417d7459c8a3fe9b92991c46f/users/4b960532407841e1886eae53db74cc96/roles/e95c2ce63e114cc19a9dda6ad21e794f` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:11.830947 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5268825-cb37-4102-9f07-7536f4ccce88 admin admin] RBAC: Authorizing `identity:create_grant(project_id=381814a417d7459c8a3fe9b92991c46f, user_id=4b960532407841e1886eae53db74cc96, role_id=e95c2ce63e114cc19a9dda6ad21e794f)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:11.840176 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5268825-cb37-4102-9f07-7536f4ccce88 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:11.875551 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 881/1763] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jun 29 11:25:11 2022] PUT /identity/v3/projects/381814a417d7459c8a3fe9b92991c46f/users/4b960532407841e1886eae53db74cc96/roles/e95c2ce63e114cc19a9dda6ad21e794f => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 11:25:11.875551 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25e9733-cff8-476c-b8df-d2b4f26d23d1 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:11.875551 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25e9733-cff8-476c-b8df-d2b4f26d23d1 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:11.875551 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25e9733-cff8-476c-b8df-d2b4f26d23d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:11.921541 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-f25e9733-cff8-476c-b8df-d2b4f26d23d1 None None] MFA Rules not processed for user `796aeeff16df4739bddec1f6dee61bfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:25:11.922842 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-f25e9733-cff8-476c-b8df-d2b4f26d23d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:25:11.928262 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:25:12.048108 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-fd7bbb8d-64fd-43a8-855a-07fb72e889f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f2f25a7608d403ea74525a7270c28a4'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:12.048108 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-fd7bbb8d-64fd-43a8-855a-07fb72e889f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f2f25a7608d403ea74525a7270c28a4'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:12.061926 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 882/1764] 10.222.0.44 () {58 vars in 1018 bytes} [Wed Jun 29 11:25:11 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:25:12.063311 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41152c91-31f3-4dba-a2aa-daab3b228d7f None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:12.063627 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41152c91-31f3-4dba-a2aa-daab3b228d7f None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:12.063905 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41152c91-31f3-4dba-a2aa-daab3b228d7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:12.127259 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-41152c91-31f3-4dba-a2aa-daab3b228d7f None None] MFA Rules not processed for user `4b960532407841e1886eae53db74cc96`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:25:12.128522 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-41152c91-31f3-4dba-a2aa-daab3b228d7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:25:12.133771 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-fd7bbb8d-64fd-43a8-855a-07fb72e889f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f2f25a7608d403ea74525a7270c28a4'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:12.149143 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-fd7bbb8d-64fd-43a8-855a-07fb72e889f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f2f25a7608d403ea74525a7270c28a4'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:12.149884 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-fd7bbb8d-64fd-43a8-855a-07fb72e889f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f2f25a7608d403ea74525a7270c28a4'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:12.175811 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 883/1765] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jun 29 11:25:11 2022] DELETE /identity/v3/users/4f2f25a7608d403ea74525a7270c28a4 => generated 0 bytes in 469 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:25:12.177526 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7bbb8d-64fd-43a8-855a-07fb72e889f1 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:12.181413 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-fd7bbb8d-64fd-43a8-855a-07fb72e889f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:25:12.223667 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 883/1766] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:25:12 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:25:12.225395 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ffe312-47f2-4d86-9885-4a84b4f34ad0 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:12.225691 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ffe312-47f2-4d86-9885-4a84b4f34ad0 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:12.227730 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ffe312-47f2-4d86-9885-4a84b4f34ad0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:12.246040 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-56ffe312-47f2-4d86-9885-4a84b4f34ad0 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:25:12.247266 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-56ffe312-47f2-4d86-9885-4a84b4f34ad0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:25:12.314548 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7bbb8d-64fd-43a8-855a-07fb72e889f1 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:12.320612 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7b2044-d409-416b-8751-b28794134c74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:12.320934 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7b2044-d409-416b-8751-b28794134c74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:12.321276 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7b2044-d409-416b-8751-b28794134c74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:12.321451 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7b2044-d409-416b-8751-b28794134c74 admin admin] PATH_INFO: `/v3/users/5aff96e0fb2c4df58217ea77b5a6fa4c` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:12.322668 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7b2044-d409-416b-8751-b28794134c74 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5aff96e0fb2c4df58217ea77b5a6fa4c)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:12.325229 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7b2044-d409-416b-8751-b28794134c74 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:12.354951 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 884/1767] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:25:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:25:12.356025 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ffe312-47f2-4d86-9885-4a84b4f34ad0 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:12.360391 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-56ffe312-47f2-4d86-9885-4a84b4f34ad0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:25:12.466943 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ffe312-47f2-4d86-9885-4a84b4f34ad0 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:12.474465 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ccd894-e4d8-4af9-8136-40eda2a0af84 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff7931b75a924394b1af733febce0834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:12.475338 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ccd894-e4d8-4af9-8136-40eda2a0af84 service cinder] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:12.475625 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ccd894-e4d8-4af9-8136-40eda2a0af84 service cinder] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:12.476498 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ccd894-e4d8-4af9-8136-40eda2a0af84 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:12.596138 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-cc7b2044-d409-416b-8751-b28794134c74 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5aff96e0fb2c4df58217ea77b5a6fa4c'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:12.599169 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-cc7b2044-d409-416b-8751-b28794134c74 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5aff96e0fb2c4df58217ea77b5a6fa4c'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:12.605070 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ccd894-e4d8-4af9-8136-40eda2a0af84 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:12.606558 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ccd894-e4d8-4af9-8136-40eda2a0af84 service cinder] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:12.640305 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 885/1768] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:25:12 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 284 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:25:12.641807 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ccd894-e4d8-4af9-8136-40eda2a0af84 service cinder] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:12.643970 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-b6ccd894-e4d8-4af9-8136-40eda2a0af84 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:25:12.713957 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-cc7b2044-d409-416b-8751-b28794134c74 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5aff96e0fb2c4df58217ea77b5a6fa4c'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:12.723730 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-cc7b2044-d409-416b-8751-b28794134c74 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5aff96e0fb2c4df58217ea77b5a6fa4c'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:12.738591 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-cc7b2044-d409-416b-8751-b28794134c74 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5aff96e0fb2c4df58217ea77b5a6fa4c'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:12.757674 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ccd894-e4d8-4af9-8136-40eda2a0af84 service cinder] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:12.762714 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcaa5c8-da66-4f7b-8bbc-a819685b0c38 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f4bb9f989e4024a5da28dced646b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:12.763727 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcaa5c8-da66-4f7b-8bbc-a819685b0c38 service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:12.764281 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcaa5c8-da66-4f7b-8bbc-a819685b0c38 service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:12.764829 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcaa5c8-da66-4f7b-8bbc-a819685b0c38 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:12.767452 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-1fcaa5c8-da66-4f7b-8bbc-a819685b0c38 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:25:12.773899 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 884/1769] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jun 29 11:25:12 2022] DELETE /identity/v3/users/5aff96e0fb2c4df58217ea77b5a6fa4c => generated 0 bytes in 597 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:25:12.776979 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579a22e6-d91f-408c-bdb0-656507a11a24 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:12.776979 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579a22e6-d91f-408c-bdb0-656507a11a24 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:12.776979 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579a22e6-d91f-408c-bdb0-656507a11a24 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:12.781013 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 885/1770] 10.222.0.44 () {54 vars in 921 bytes} [Wed Jun 29 11:25:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:25:12.781013 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579a22e6-d91f-408c-bdb0-656507a11a24 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:12.781013 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-579a22e6-d91f-408c-bdb0-656507a11a24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:25:12.794307 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcaa5c8-da66-4f7b-8bbc-a819685b0c38 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:12.796159 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcaa5c8-da66-4f7b-8bbc-a819685b0c38 service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:12.810216 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 886/1771] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:25:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:25:12.811905 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc92ec81-c0fe-4c20-a0d6-ee8b936945b5 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:12.815233 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc92ec81-c0fe-4c20-a0d6-ee8b936945b5 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:12.815233 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc92ec81-c0fe-4c20-a0d6-ee8b936945b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:12.876231 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-bc92ec81-c0fe-4c20-a0d6-ee8b936945b5 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:25:12.879000 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-bc92ec81-c0fe-4c20-a0d6-ee8b936945b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:25:12.942806 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579a22e6-d91f-408c-bdb0-656507a11a24 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:12.958663 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6ec5f3-6a64-4263-a14b-3e7889830a43 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '796aeeff16df4739bddec1f6dee61bfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:12.959375 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6ec5f3-6a64-4263-a14b-3e7889830a43 service nova] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:12.959667 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6ec5f3-6a64-4263-a14b-3e7889830a43 service nova] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:12.962934 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6ec5f3-6a64-4263-a14b-3e7889830a43 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:12.965385 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-2f6ec5f3-6a64-4263-a14b-3e7889830a43 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:25:12.990238 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6ec5f3-6a64-4263-a14b-3e7889830a43 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:12.992298 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6ec5f3-6a64-4263-a14b-3e7889830a43 service nova] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:13.007792 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 886/1772] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:25:12 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:25:13.009218 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886d64ed-a480-4b90-a7b4-dea2424e778d None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:13.009628 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886d64ed-a480-4b90-a7b4-dea2424e778d None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:13.010047 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886d64ed-a480-4b90-a7b4-dea2424e778d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:13.017321 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 887/1773] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:25:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:25:13.020071 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc92ec81-c0fe-4c20-a0d6-ee8b936945b5 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:13.021662 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:25:13.021788 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-bc92ec81-c0fe-4c20-a0d6-ee8b936945b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:25:13.058255 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-886d64ed-a480-4b90-a7b4-dea2424e778d None None] MFA Rules not processed for user `796aeeff16df4739bddec1f6dee61bfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:25:13.059716 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-886d64ed-a480-4b90-a7b4-dea2424e778d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:25:13.083484 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 887/1774] 10.222.0.44 () {58 vars in 1018 bytes} [Wed Jun 29 11:25:13 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:25:13.124536 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc92ec81-c0fe-4c20-a0d6-ee8b936945b5 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:13.146869 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d441affa-d771-47d2-bd99-dd4b4b3129d4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff7931b75a924394b1af733febce0834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:13.150799 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886d64ed-a480-4b90-a7b4-dea2424e778d None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:13.151897 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:25:13.152576 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d441affa-d771-47d2-bd99-dd4b4b3129d4 service cinder] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:13.153150 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-886d64ed-a480-4b90-a7b4-dea2424e778d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:25:13.154487 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d441affa-d771-47d2-bd99-dd4b4b3129d4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:13.155062 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d441affa-d771-47d2-bd99-dd4b4b3129d4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:13.159335 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-d441affa-d771-47d2-bd99-dd4b4b3129d4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:25:13.227463 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:25:13.250055 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886d64ed-a480-4b90-a7b4-dea2424e778d None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:13.255079 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050ac67e-8936-4623-bec4-11c108ffd124 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f4bb9f989e4024a5da28dced646b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:13.260287 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050ac67e-8936-4623-bec4-11c108ffd124 service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:13.261094 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050ac67e-8936-4623-bec4-11c108ffd124 service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:13.264699 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050ac67e-8936-4623-bec4-11c108ffd124 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:13.276365 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050ac67e-8936-4623-bec4-11c108ffd124 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:13.278549 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050ac67e-8936-4623-bec4-11c108ffd124 service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:13.282483 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d441affa-d771-47d2-bd99-dd4b4b3129d4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:13.284266 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d441affa-d771-47d2-bd99-dd4b4b3129d4 service cinder] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:13.294205 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 888/1775] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:25:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 11:25:13.295202 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050ac67e-8936-4623-bec4-11c108ffd124 service neutron] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:13.300576 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 888/1776] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:25:13 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 283 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:25:13.316390 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050ac67e-8936-4623-bec4-11c108ffd124 service neutron] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:13.324568 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef5abb7-44d9-4466-bf9b-72439da9def2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '796aeeff16df4739bddec1f6dee61bfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:13.325386 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef5abb7-44d9-4466-bf9b-72439da9def2 service nova] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:13.325800 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef5abb7-44d9-4466-bf9b-72439da9def2 service nova] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:13.326148 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef5abb7-44d9-4466-bf9b-72439da9def2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:13.338728 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef5abb7-44d9-4466-bf9b-72439da9def2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:13.340456 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef5abb7-44d9-4466-bf9b-72439da9def2 service nova] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:13.355857 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 889/1777] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:25:13 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:25:13.960350 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:25:14.037801 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d441affa-d771-47d2-bd99-dd4b4b3129d4 service cinder] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:14.039894 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-d441affa-d771-47d2-bd99-dd4b4b3129d4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:25:14.081328 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d441affa-d771-47d2-bd99-dd4b4b3129d4 service cinder] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:14.083900 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=80,keystone:INSERT=5,keystone:DELETE=15 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:25:14.089332 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16c0a3d-4b54-4a32-a2f7-90cf64f71fa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:14.089332 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16c0a3d-4b54-4a32-a2f7-90cf64f71fa8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:14.089332 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16c0a3d-4b54-4a32-a2f7-90cf64f71fa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:14.089332 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16c0a3d-4b54-4a32-a2f7-90cf64f71fa8 admin admin] PATH_INFO: `/v3/projects/fd6481343c5c420a9d3ced7e316b7ca7` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:14.090110 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16c0a3d-4b54-4a32-a2f7-90cf64f71fa8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd6481343c5c420a9d3ced7e316b7ca7)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:14.091686 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16c0a3d-4b54-4a32-a2f7-90cf64f71fa8 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:14.193826 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef5abb7-44d9-4466-bf9b-72439da9def2 service nova] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:14.195286 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-4ef5abb7-44d9-4466-bf9b-72439da9def2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:25:14.205868 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-f16c0a3d-4b54-4a32-a2f7-90cf64f71fa8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd6481343c5c420a9d3ced7e316b7ca7', 'request_id': 'req-f16c0a3d-4b54-4a32-a2f7-90cf64f71fa8'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:14.208849 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef5abb7-44d9-4466-bf9b-72439da9def2 service nova] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:14.257153 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-f16c0a3d-4b54-4a32-a2f7-90cf64f71fa8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd6481343c5c420a9d3ced7e316b7ca7', 'request_id': 'req-f16c0a3d-4b54-4a32-a2f7-90cf64f71fa8'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:14.265220 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-f16c0a3d-4b54-4a32-a2f7-90cf64f71fa8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd6481343c5c420a9d3ced7e316b7ca7', 'request_id': 'req-f16c0a3d-4b54-4a32-a2f7-90cf64f71fa8'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:14.275767 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09623eec-91f7-4d9e-84b0-d4ea38faa39e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:14.276585 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-f16c0a3d-4b54-4a32-a2f7-90cf64f71fa8 admin admin] The token cache is being invalidate because project fd6481343c5c420a9d3ced7e316b7ca7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 11:25:14.276695 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09623eec-91f7-4d9e-84b0-d4ea38faa39e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:14.276873 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-f16c0a3d-4b54-4a32-a2f7-90cf64f71fa8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:14.277011 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09623eec-91f7-4d9e-84b0-d4ea38faa39e admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:14.277221 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09623eec-91f7-4d9e-84b0-d4ea38faa39e admin admin] PATH_INFO: `/v3/projects/a0273d33a17f4d74a1ec01c46107a6e4` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:14.277945 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-f16c0a3d-4b54-4a32-a2f7-90cf64f71fa8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:14.278411 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09623eec-91f7-4d9e-84b0-d4ea38faa39e admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0273d33a17f4d74a1ec01c46107a6e4)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:14.279648 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 889/1778] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jun 29 11:25:14 2022] DELETE /identity/v3/projects/fd6481343c5c420a9d3ced7e316b7ca7 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:25:14.280357 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09623eec-91f7-4d9e-84b0-d4ea38faa39e admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:14.283568 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16c0a3d-4b54-4a32-a2f7-90cf64f71fa8 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:14.285068 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-f16c0a3d-4b54-4a32-a2f7-90cf64f71fa8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:25:14.299229 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16c0a3d-4b54-4a32-a2f7-90cf64f71fa8 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:14.303613 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c5539d-2e9e-4f74-869b-a773e42caba1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:14.308219 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c5539d-2e9e-4f74-869b-a773e42caba1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:14.308219 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c5539d-2e9e-4f74-869b-a773e42caba1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:14.308219 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c5539d-2e9e-4f74-869b-a773e42caba1 admin admin] PATH_INFO: `/v3/users/e7c5afe798434bbf8f6165152065482a` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:14.308219 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c5539d-2e9e-4f74-869b-a773e42caba1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e7c5afe798434bbf8f6165152065482a)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:14.308756 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c5539d-2e9e-4f74-869b-a773e42caba1 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:14.395043 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-09623eec-91f7-4d9e-84b0-d4ea38faa39e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0273d33a17f4d74a1ec01c46107a6e4', 'request_id': 'req-09623eec-91f7-4d9e-84b0-d4ea38faa39e'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:14.402017 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-09623eec-91f7-4d9e-84b0-d4ea38faa39e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0273d33a17f4d74a1ec01c46107a6e4', 'request_id': 'req-09623eec-91f7-4d9e-84b0-d4ea38faa39e'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:14.407988 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-09623eec-91f7-4d9e-84b0-d4ea38faa39e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0273d33a17f4d74a1ec01c46107a6e4', 'request_id': 'req-09623eec-91f7-4d9e-84b0-d4ea38faa39e'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:14.471861 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-09623eec-91f7-4d9e-84b0-d4ea38faa39e admin admin] The token cache is being invalidate because project a0273d33a17f4d74a1ec01c46107a6e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 11:25:14.472166 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-09623eec-91f7-4d9e-84b0-d4ea38faa39e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:14.472808 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-09623eec-91f7-4d9e-84b0-d4ea38faa39e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:14.474812 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 890/1779] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jun 29 11:25:14 2022] DELETE /identity/v3/projects/a0273d33a17f4d74a1ec01c46107a6e4 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 11:25:14.478869 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09623eec-91f7-4d9e-84b0-d4ea38faa39e admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:14.480307 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-09623eec-91f7-4d9e-84b0-d4ea38faa39e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:25:14.517195 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-70c5539d-2e9e-4f74-869b-a773e42caba1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e7c5afe798434bbf8f6165152065482a'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:14.524576 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:25:14.551319 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09623eec-91f7-4d9e-84b0-d4ea38faa39e admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:14.557691 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a962519-26ec-43a9-90c3-1505798ebb63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:14.567355 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a962519-26ec-43a9-90c3-1505798ebb63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:14.567355 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a962519-26ec-43a9-90c3-1505798ebb63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:14.567355 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a962519-26ec-43a9-90c3-1505798ebb63 admin admin] PATH_INFO: `/v3/users/f015bb491e3b42ca928d05e49ecea011` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:14.567355 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a962519-26ec-43a9-90c3-1505798ebb63 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f015bb491e3b42ca928d05e49ecea011)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:14.567355 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a962519-26ec-43a9-90c3-1505798ebb63 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:14.639562 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-70c5539d-2e9e-4f74-869b-a773e42caba1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e7c5afe798434bbf8f6165152065482a'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:14.640224 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-70c5539d-2e9e-4f74-869b-a773e42caba1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e7c5afe798434bbf8f6165152065482a'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:14.653159 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-70c5539d-2e9e-4f74-869b-a773e42caba1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e7c5afe798434bbf8f6165152065482a'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:14.653783 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-70c5539d-2e9e-4f74-869b-a773e42caba1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e7c5afe798434bbf8f6165152065482a'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:14.665862 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 890/1780] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jun 29 11:25:14 2022] DELETE /identity/v3/users/e7c5afe798434bbf8f6165152065482a => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 11:25:14.768342 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-4a962519-26ec-43a9-90c3-1505798ebb63 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f015bb491e3b42ca928d05e49ecea011'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:14.768933 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-4a962519-26ec-43a9-90c3-1505798ebb63 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f015bb491e3b42ca928d05e49ecea011'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:14.839547 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-4a962519-26ec-43a9-90c3-1505798ebb63 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f015bb491e3b42ca928d05e49ecea011'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:14.847227 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-4a962519-26ec-43a9-90c3-1505798ebb63 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f015bb491e3b42ca928d05e49ecea011'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:14.848034 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-4a962519-26ec-43a9-90c3-1505798ebb63 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f015bb491e3b42ca928d05e49ecea011'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:14.869154 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 891/1781] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jun 29 11:25:14 2022] DELETE /identity/v3/users/f015bb491e3b42ca928d05e49ecea011 => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 11:25:15.099017 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c5539d-2e9e-4f74-869b-a773e42caba1 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:15.100758 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-70c5539d-2e9e-4f74-869b-a773e42caba1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:25:15.140527 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:25:15.199129 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c5539d-2e9e-4f74-869b-a773e42caba1 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:15.267150 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f766fe4d-1957-4fe7-acc1-c3947ca91e7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:15.268490 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f766fe4d-1957-4fe7-acc1-c3947ca91e7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:15.268787 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f766fe4d-1957-4fe7-acc1-c3947ca91e7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:15.278125 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f766fe4d-1957-4fe7-acc1-c3947ca91e7f admin admin] PATH_INFO: `/v3/projects/c5f1a2c361284f84a445b0991532d9ee` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:15.278125 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f766fe4d-1957-4fe7-acc1-c3947ca91e7f admin admin] RBAC: Authorizing `identity:delete_project(project_id=c5f1a2c361284f84a445b0991532d9ee)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:15.278125 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f766fe4d-1957-4fe7-acc1-c3947ca91e7f admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:15.371357 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-f766fe4d-1957-4fe7-acc1-c3947ca91e7f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c5f1a2c361284f84a445b0991532d9ee', 'request_id': 'req-f766fe4d-1957-4fe7-acc1-c3947ca91e7f'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:15.437943 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-f766fe4d-1957-4fe7-acc1-c3947ca91e7f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c5f1a2c361284f84a445b0991532d9ee', 'request_id': 'req-f766fe4d-1957-4fe7-acc1-c3947ca91e7f'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:15.447589 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-f766fe4d-1957-4fe7-acc1-c3947ca91e7f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c5f1a2c361284f84a445b0991532d9ee', 'request_id': 'req-f766fe4d-1957-4fe7-acc1-c3947ca91e7f'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:15.453873 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a962519-26ec-43a9-90c3-1505798ebb63 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:15.456270 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-4a962519-26ec-43a9-90c3-1505798ebb63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:25:15.480447 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-f766fe4d-1957-4fe7-acc1-c3947ca91e7f admin admin] The token cache is being invalidate because project c5f1a2c361284f84a445b0991532d9ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 11:25:15.480447 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-f766fe4d-1957-4fe7-acc1-c3947ca91e7f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:15.480447 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-f766fe4d-1957-4fe7-acc1-c3947ca91e7f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:15.483013 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 891/1782] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jun 29 11:25:15 2022] DELETE /identity/v3/projects/c5f1a2c361284f84a445b0991532d9ee => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:25:15.487891 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93aa642a-ca9c-49ed-8c02-b64edfd49d80 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:15.488280 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93aa642a-ca9c-49ed-8c02-b64edfd49d80 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:15.488729 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93aa642a-ca9c-49ed-8c02-b64edfd49d80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:15.493241 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a962519-26ec-43a9-90c3-1505798ebb63 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:15.512544 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-93aa642a-ca9c-49ed-8c02-b64edfd49d80 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:25:15.513898 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-93aa642a-ca9c-49ed-8c02-b64edfd49d80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:25:15.579288 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbd0720-a07f-4f07-bdcc-3ebba9c673dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:15.580300 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbd0720-a07f-4f07-bdcc-3ebba9c673dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:15.588184 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbd0720-a07f-4f07-bdcc-3ebba9c673dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:15.588184 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbd0720-a07f-4f07-bdcc-3ebba9c673dc admin admin] PATH_INFO: `/v3/projects/286480ac54ae4066bbf33c25f59c0cdb` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:15.588184 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbd0720-a07f-4f07-bdcc-3ebba9c673dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=286480ac54ae4066bbf33c25f59c0cdb)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:15.588184 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbd0720-a07f-4f07-bdcc-3ebba9c673dc admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:15.611490 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 892/1783] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:25:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:25:15.624912 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a115167b-d642-4acf-b358-c27daddf69be None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:15.625215 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a115167b-d642-4acf-b358-c27daddf69be None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:15.625508 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a115167b-d642-4acf-b358-c27daddf69be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:15.666573 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-a115167b-d642-4acf-b358-c27daddf69be None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:25:15.668378 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-a115167b-d642-4acf-b358-c27daddf69be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:25:15.676780 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 893/1784] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:25:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:25:15.681532 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a115167b-d642-4acf-b358-c27daddf69be None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:15.690669 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a115167b-d642-4acf-b358-c27daddf69be None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:15.695641 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303c05f1-6f2b-4f8d-933a-81f8b29d6918 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:15.696374 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303c05f1-6f2b-4f8d-933a-81f8b29d6918 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:15.696705 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303c05f1-6f2b-4f8d-933a-81f8b29d6918 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:15.696987 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303c05f1-6f2b-4f8d-933a-81f8b29d6918 admin admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:15.697810 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303c05f1-6f2b-4f8d-933a-81f8b29d6918 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:25:15.698019 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303c05f1-6f2b-4f8d-933a-81f8b29d6918 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:15.699655 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303c05f1-6f2b-4f8d-933a-81f8b29d6918 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:15.713768 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 894/1785] 10.222.0.44 () {58 vars in 1152 bytes} [Wed Jun 29 11:25:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:25:15.716647 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303c05f1-6f2b-4f8d-933a-81f8b29d6918 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:15.726965 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303c05f1-6f2b-4f8d-933a-81f8b29d6918 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:15.731594 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8073bec6-b69c-41d9-8611-d56ce76c9096 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:15.732262 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8073bec6-b69c-41d9-8611-d56ce76c9096 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:15.732543 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8073bec6-b69c-41d9-8611-d56ce76c9096 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:15.732817 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8073bec6-b69c-41d9-8611-d56ce76c9096 admin admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:15.733864 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8073bec6-b69c-41d9-8611-d56ce76c9096 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:15.737397 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8073bec6-b69c-41d9-8611-d56ce76c9096 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:15.750145 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-3dbd0720-a07f-4f07-bdcc-3ebba9c673dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '286480ac54ae4066bbf33c25f59c0cdb', 'request_id': 'req-3dbd0720-a07f-4f07-bdcc-3ebba9c673dc'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:15.756589 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-3dbd0720-a07f-4f07-bdcc-3ebba9c673dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '286480ac54ae4066bbf33c25f59c0cdb', 'request_id': 'req-3dbd0720-a07f-4f07-bdcc-3ebba9c673dc'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:15.761946 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-3dbd0720-a07f-4f07-bdcc-3ebba9c673dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '286480ac54ae4066bbf33c25f59c0cdb', 'request_id': 'req-3dbd0720-a07f-4f07-bdcc-3ebba9c673dc'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:15.839683 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-3dbd0720-a07f-4f07-bdcc-3ebba9c673dc admin admin] The token cache is being invalidate because project 286480ac54ae4066bbf33c25f59c0cdb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 11:25:15.839910 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-3dbd0720-a07f-4f07-bdcc-3ebba9c673dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:15.840540 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-3dbd0720-a07f-4f07-bdcc-3ebba9c673dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:15.845855 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 895/1786] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jun 29 11:25:15 2022] POST /identity/v3/projects => generated 379 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 11:25:15.845991 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 892/1787] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jun 29 11:25:15 2022] DELETE /identity/v3/projects/286480ac54ae4066bbf33c25f59c0cdb => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:25:15.846781 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbd0720-a07f-4f07-bdcc-3ebba9c673dc admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:15.847094 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b904222a-9d5a-441d-8a64-ea37a588e5b0 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:15.847384 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b904222a-9d5a-441d-8a64-ea37a588e5b0 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:15.847655 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b904222a-9d5a-441d-8a64-ea37a588e5b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:15.848600 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-3dbd0720-a07f-4f07-bdcc-3ebba9c673dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:25:15.864934 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-b904222a-9d5a-441d-8a64-ea37a588e5b0 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:25:15.866140 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-b904222a-9d5a-441d-8a64-ea37a588e5b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:25:15.918658 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbd0720-a07f-4f07-bdcc-3ebba9c673dc admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:15.933123 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 896/1788] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:25:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 11:25:15.934036 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:25:15.942144 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262adcb3-466f-4202-aef0-5d0371d51a86 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:15.942443 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262adcb3-466f-4202-aef0-5d0371d51a86 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:15.942719 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262adcb3-466f-4202-aef0-5d0371d51a86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:15.960989 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-262adcb3-466f-4202-aef0-5d0371d51a86 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:25:15.962202 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-262adcb3-466f-4202-aef0-5d0371d51a86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:25:15.970710 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 897/1789] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:25:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 11:25:15.974684 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262adcb3-466f-4202-aef0-5d0371d51a86 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:15.978962 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2f8751-132a-427a-a2a0-4e847cdc94c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:15.980041 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2f8751-132a-427a-a2a0-4e847cdc94c8 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:15.980323 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2f8751-132a-427a-a2a0-4e847cdc94c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:15.980595 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2f8751-132a-427a-a2a0-4e847cdc94c8 admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:15.981421 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2f8751-132a-427a-a2a0-4e847cdc94c8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:15.983328 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2f8751-132a-427a-a2a0-4e847cdc94c8 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:15.983870 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262adcb3-466f-4202-aef0-5d0371d51a86 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:15.988475 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20337688-8035-492d-aedc-06ad9fc624fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:15.989117 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20337688-8035-492d-aedc-06ad9fc624fd admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:15.989431 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20337688-8035-492d-aedc-06ad9fc624fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:15.989708 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20337688-8035-492d-aedc-06ad9fc624fd admin admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:15.990427 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20337688-8035-492d-aedc-06ad9fc624fd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:25:15.990741 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20337688-8035-492d-aedc-06ad9fc624fd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:15.993013 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20337688-8035-492d-aedc-06ad9fc624fd admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:16.004626 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 898/1790] 10.222.0.44 () {58 vars in 1152 bytes} [Wed Jun 29 11:25:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 11:25:16.009596 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20337688-8035-492d-aedc-06ad9fc624fd admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:16.018977 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20337688-8035-492d-aedc-06ad9fc624fd admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:16.023437 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7794e4cd-d175-4525-a78a-b85336e7e1d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:16.024080 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7794e4cd-d175-4525-a78a-b85336e7e1d4 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:16.024363 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7794e4cd-d175-4525-a78a-b85336e7e1d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:16.024638 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7794e4cd-d175-4525-a78a-b85336e7e1d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:16.025312 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7794e4cd-d175-4525-a78a-b85336e7e1d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:16.027236 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7794e4cd-d175-4525-a78a-b85336e7e1d4 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:16.062822 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 893/1791] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jun 29 11:25:15 2022] POST /identity/v3/users => generated 290 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 11:25:16.067000 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2f8751-132a-427a-a2a0-4e847cdc94c8 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:16.077672 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2f8751-132a-427a-a2a0-4e847cdc94c8 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:16.082922 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03795a75-0571-48b4-b0c8-ffabd498fb4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:16.083588 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03795a75-0571-48b4-b0c8-ffabd498fb4d admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:16.087793 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03795a75-0571-48b4-b0c8-ffabd498fb4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:16.087793 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03795a75-0571-48b4-b0c8-ffabd498fb4d admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:16.087793 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03795a75-0571-48b4-b0c8-ffabd498fb4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:16.087793 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03795a75-0571-48b4-b0c8-ffabd498fb4d admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:16.096956 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 894/1792] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:25:16 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 11:25:16.101353 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03795a75-0571-48b4-b0c8-ffabd498fb4d admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:16.110850 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03795a75-0571-48b4-b0c8-ffabd498fb4d admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:16.115726 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0b8a80-67c8-40cd-ace9-2f9e691331fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:16.116450 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0b8a80-67c8-40cd-ace9-2f9e691331fc admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:16.117290 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0b8a80-67c8-40cd-ace9-2f9e691331fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:16.117569 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0b8a80-67c8-40cd-ace9-2f9e691331fc admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:16.118437 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0b8a80-67c8-40cd-ace9-2f9e691331fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:16.120067 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0b8a80-67c8-40cd-ace9-2f9e691331fc admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:16.137230 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 895/1793] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:25:16 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 11:25:16.138269 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0b8a80-67c8-40cd-ace9-2f9e691331fc admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:16.142616 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 899/1794] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jun 29 11:25:16 2022] POST /identity/v3/projects => generated 385 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 11:25:16.146973 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7794e4cd-d175-4525-a78a-b85336e7e1d4 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:16.158578 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0b8a80-67c8-40cd-ace9-2f9e691331fc admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:16.245806 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7794e4cd-d175-4525-a78a-b85336e7e1d4 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:16.251261 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d27f22-78fc-4e6e-a6c7-f0db222550ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:16.251962 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d27f22-78fc-4e6e-a6c7-f0db222550ed admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:16.253224 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d27f22-78fc-4e6e-a6c7-f0db222550ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:16.253515 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d27f22-78fc-4e6e-a6c7-f0db222550ed admin admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:16.254543 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d27f22-78fc-4e6e-a6c7-f0db222550ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:16.257157 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d27f22-78fc-4e6e-a6c7-f0db222550ed admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:16.308215 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbe9113-4925-4f7d-afba-68fa90545a8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:16.309128 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbe9113-4925-4f7d-afba-68fa90545a8f admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:16.309418 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbe9113-4925-4f7d-afba-68fa90545a8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:16.309697 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbe9113-4925-4f7d-afba-68fa90545a8f admin admin] PATH_INFO: `/v3/projects/7cb53b7fa6cd47358735e8ba3907e8b6/users/979606c20ef3442e808b191ce497df58/roles/e1c88eebe911432cab6a07fbeb84653b` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:16.326407 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbe9113-4925-4f7d-afba-68fa90545a8f admin admin] RBAC: Authorizing `identity:create_grant(project_id=7cb53b7fa6cd47358735e8ba3907e8b6, user_id=979606c20ef3442e808b191ce497df58, role_id=e1c88eebe911432cab6a07fbeb84653b)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:16.329154 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbe9113-4925-4f7d-afba-68fa90545a8f admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:16.351701 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 900/1795] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jun 29 11:25:16 2022] POST /identity/v3/users => generated 293 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 11:25:16.356744 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d27f22-78fc-4e6e-a6c7-f0db222550ed admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:16.378378 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d27f22-78fc-4e6e-a6c7-f0db222550ed admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:16.389164 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50caec73-5e82-46aa-882f-44d5fe5bfd3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:16.389926 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50caec73-5e82-46aa-882f-44d5fe5bfd3a admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:16.390277 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50caec73-5e82-46aa-882f-44d5fe5bfd3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:16.390627 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50caec73-5e82-46aa-882f-44d5fe5bfd3a admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:16.397180 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50caec73-5e82-46aa-882f-44d5fe5bfd3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:16.399219 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50caec73-5e82-46aa-882f-44d5fe5bfd3a admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:16.415192 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 901/1796] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:25:16 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 11:25:16.418906 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50caec73-5e82-46aa-882f-44d5fe5bfd3a admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:16.430738 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50caec73-5e82-46aa-882f-44d5fe5bfd3a admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:16.437157 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 896/1797] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jun 29 11:25:16 2022] PUT /identity/v3/projects/7cb53b7fa6cd47358735e8ba3907e8b6/users/979606c20ef3442e808b191ce497df58/roles/e1c88eebe911432cab6a07fbeb84653b => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:25:16.453772 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6befd637-9c63-444f-be67-2c2ab0535077 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:16.454133 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6befd637-9c63-444f-be67-2c2ab0535077 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:16.454733 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6befd637-9c63-444f-be67-2c2ab0535077 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:16.489728 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-6befd637-9c63-444f-be67-2c2ab0535077 None None] MFA Rules not processed for user `979606c20ef3442e808b191ce497df58`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:25:16.491439 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-6befd637-9c63-444f-be67-2c2ab0535077 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:25:16.513088 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed2e1e5-81d6-499c-94f6-5131e35cd7c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:16.513768 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed2e1e5-81d6-499c-94f6-5131e35cd7c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:16.514053 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed2e1e5-81d6-499c-94f6-5131e35cd7c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:16.514331 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed2e1e5-81d6-499c-94f6-5131e35cd7c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:16.515064 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed2e1e5-81d6-499c-94f6-5131e35cd7c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:16.516767 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed2e1e5-81d6-499c-94f6-5131e35cd7c5 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:16.531808 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 902/1798] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:25:16 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 11:25:16.532590 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed2e1e5-81d6-499c-94f6-5131e35cd7c5 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:16.544207 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed2e1e5-81d6-499c-94f6-5131e35cd7c5 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:16.549594 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f43b259-86a7-4cc0-baf0-bccb9cb5ceb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:16.550571 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f43b259-86a7-4cc0-baf0-bccb9cb5ceb2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:16.550912 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f43b259-86a7-4cc0-baf0-bccb9cb5ceb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:16.551191 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f43b259-86a7-4cc0-baf0-bccb9cb5ceb2 admin admin] PATH_INFO: `/v3/projects/d5024ac6c5054182bf1d15a59df3f991/users/066ab897f94045da8d6aae855560bf4f/roles/e1c88eebe911432cab6a07fbeb84653b` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:16.573775 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 897/1799] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:25:16 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:25:16.576420 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f43b259-86a7-4cc0-baf0-bccb9cb5ceb2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5024ac6c5054182bf1d15a59df3f991, user_id=066ab897f94045da8d6aae855560bf4f, role_id=e1c88eebe911432cab6a07fbeb84653b)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:16.577247 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6befd637-9c63-444f-be67-2c2ab0535077 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:16.579650 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f43b259-86a7-4cc0-baf0-bccb9cb5ceb2 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:16.589311 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6befd637-9c63-444f-be67-2c2ab0535077 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:16.596355 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f681723-86f7-4457-b8fa-74b1b6ff4228 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:16.597301 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f681723-86f7-4457-b8fa-74b1b6ff4228 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:16.607284 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f681723-86f7-4457-b8fa-74b1b6ff4228 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:16.607284 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f681723-86f7-4457-b8fa-74b1b6ff4228 admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:16.607284 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f681723-86f7-4457-b8fa-74b1b6ff4228 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:16.607284 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f681723-86f7-4457-b8fa-74b1b6ff4228 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:16.672150 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 903/1800] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jun 29 11:25:16 2022] PUT /identity/v3/projects/d5024ac6c5054182bf1d15a59df3f991/users/066ab897f94045da8d6aae855560bf4f/roles/e1c88eebe911432cab6a07fbeb84653b => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:25:16.685351 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c286cff-e89b-43a7-9f74-8d855be7a7d5 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:16.685351 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c286cff-e89b-43a7-9f74-8d855be7a7d5 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:16.685351 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c286cff-e89b-43a7-9f74-8d855be7a7d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:16.718280 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-4c286cff-e89b-43a7-9f74-8d855be7a7d5 None None] MFA Rules not processed for user `066ab897f94045da8d6aae855560bf4f`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:25:16.720220 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-4c286cff-e89b-43a7-9f74-8d855be7a7d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:25:16.736667 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 898/1801] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jun 29 11:25:16 2022] POST /identity/v3/projects => generated 381 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 11:25:16.738155 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f681723-86f7-4457-b8fa-74b1b6ff4228 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:16.817628 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f681723-86f7-4457-b8fa-74b1b6ff4228 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:16.838071 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9593bf3c-9955-46e7-9298-cafdd9b9b52d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:16.839592 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9593bf3c-9955-46e7-9298-cafdd9b9b52d admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:16.840221 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9593bf3c-9955-46e7-9298-cafdd9b9b52d admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:16.845151 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9593bf3c-9955-46e7-9298-cafdd9b9b52d admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:16.845151 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9593bf3c-9955-46e7-9298-cafdd9b9b52d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:16.846636 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9593bf3c-9955-46e7-9298-cafdd9b9b52d admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:16.854675 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 904/1802] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:25:16 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 11:25:16.859778 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c286cff-e89b-43a7-9f74-8d855be7a7d5 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:16.881878 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c286cff-e89b-43a7-9f74-8d855be7a7d5 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:16.887373 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:25:16.911404 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70785784-d7a7-4986-9d2d-2ea00fb59639 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:16.914221 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70785784-d7a7-4986-9d2d-2ea00fb59639 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:16.914514 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70785784-d7a7-4986-9d2d-2ea00fb59639 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:16.916001 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70785784-d7a7-4986-9d2d-2ea00fb59639 admin admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:16.917983 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70785784-d7a7-4986-9d2d-2ea00fb59639 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:16.922580 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70785784-d7a7-4986-9d2d-2ea00fb59639 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:16.938200 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 899/1803] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jun 29 11:25:16 2022] POST /identity/v3/users => generated 297 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 11:25:16.942546 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9593bf3c-9955-46e7-9298-cafdd9b9b52d admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:16.961335 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9593bf3c-9955-46e7-9298-cafdd9b9b52d admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:16.975959 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf778cb6-5953-43fc-8b31-d4b296ea9220 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:16.977023 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf778cb6-5953-43fc-8b31-d4b296ea9220 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:16.977308 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf778cb6-5953-43fc-8b31-d4b296ea9220 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:16.978031 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf778cb6-5953-43fc-8b31-d4b296ea9220 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:16.979030 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf778cb6-5953-43fc-8b31-d4b296ea9220 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:16.980747 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf778cb6-5953-43fc-8b31-d4b296ea9220 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:16.995935 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 900/1804] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:25:16 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 11:25:17.000362 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf778cb6-5953-43fc-8b31-d4b296ea9220 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:17.013446 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf778cb6-5953-43fc-8b31-d4b296ea9220 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:17.020465 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef369396-d166-4623-98f6-d2667979377a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:17.021904 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef369396-d166-4623-98f6-d2667979377a admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:17.022456 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef369396-d166-4623-98f6-d2667979377a admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:17.023575 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef369396-d166-4623-98f6-d2667979377a admin admin] PATH_INFO: `/v3/projects/e00c5a05fe4c43fa9af22772d290cd20/users/1bee23d32a3f49b78d485c60dafce049/roles/e95c2ce63e114cc19a9dda6ad21e794f` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:17.037133 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 905/1805] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jun 29 11:25:16 2022] POST /identity/v3/projects => generated 387 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:25:17.041043 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70785784-d7a7-4986-9d2d-2ea00fb59639 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:17.052841 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef369396-d166-4623-98f6-d2667979377a admin admin] RBAC: Authorizing `identity:create_grant(project_id=e00c5a05fe4c43fa9af22772d290cd20, user_id=1bee23d32a3f49b78d485c60dafce049, role_id=e95c2ce63e114cc19a9dda6ad21e794f)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:17.056960 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef369396-d166-4623-98f6-d2667979377a admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:17.118923 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 901/1806] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jun 29 11:25:16 2022] PUT /identity/v3/projects/e00c5a05fe4c43fa9af22772d290cd20/users/1bee23d32a3f49b78d485c60dafce049/roles/e95c2ce63e114cc19a9dda6ad21e794f => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:25:17.122358 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70785784-d7a7-4986-9d2d-2ea00fb59639 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:17.122700 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef369396-d166-4623-98f6-d2667979377a admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:17.184863 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec5c109-c5f4-4a7c-9570-061e42416325 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:17.185548 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec5c109-c5f4-4a7c-9570-061e42416325 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:17.185854 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec5c109-c5f4-4a7c-9570-061e42416325 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:17.186130 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec5c109-c5f4-4a7c-9570-061e42416325 admin admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:17.186825 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec5c109-c5f4-4a7c-9570-061e42416325 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:17.188853 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec5c109-c5f4-4a7c-9570-061e42416325 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:17.190189 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef369396-d166-4623-98f6-d2667979377a admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:17.195540 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3c759c-9eb8-4657-b6cc-d1dd449ba8bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:17.196398 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3c759c-9eb8-4657-b6cc-d1dd449ba8bf admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:17.196660 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3c759c-9eb8-4657-b6cc-d1dd449ba8bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:17.196992 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3c759c-9eb8-4657-b6cc-d1dd449ba8bf admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:17.197672 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3c759c-9eb8-4657-b6cc-d1dd449ba8bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:17.199340 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3c759c-9eb8-4657-b6cc-d1dd449ba8bf admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:17.209438 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 902/1807] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:25:17 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 11:25:17.214107 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3c759c-9eb8-4657-b6cc-d1dd449ba8bf admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:17.223732 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3c759c-9eb8-4657-b6cc-d1dd449ba8bf admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:17.228291 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915a6e30-c92f-4291-8020-1aa09082cdbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:17.229130 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915a6e30-c92f-4291-8020-1aa09082cdbf admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:17.229409 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915a6e30-c92f-4291-8020-1aa09082cdbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:17.230816 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915a6e30-c92f-4291-8020-1aa09082cdbf admin admin] PATH_INFO: `/v3/projects/e00c5a05fe4c43fa9af22772d290cd20/users/1bee23d32a3f49b78d485c60dafce049/roles/e95c2ce63e114cc19a9dda6ad21e794f` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:17.231591 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915a6e30-c92f-4291-8020-1aa09082cdbf admin admin] RBAC: Authorizing `identity:create_grant(project_id=e00c5a05fe4c43fa9af22772d290cd20, user_id=1bee23d32a3f49b78d485c60dafce049, role_id=e95c2ce63e114cc19a9dda6ad21e794f)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:17.234201 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915a6e30-c92f-4291-8020-1aa09082cdbf admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:17.244846 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 903/1808] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jun 29 11:25:17 2022] PUT /identity/v3/projects/e00c5a05fe4c43fa9af22772d290cd20/users/1bee23d32a3f49b78d485c60dafce049/roles/e95c2ce63e114cc19a9dda6ad21e794f => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 11:25:17.254741 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd30adb3-5f4d-4dd9-b0be-e9b8546233cb None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:17.255033 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd30adb3-5f4d-4dd9-b0be-e9b8546233cb None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:17.255318 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd30adb3-5f4d-4dd9-b0be-e9b8546233cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:17.281529 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-dd30adb3-5f4d-4dd9-b0be-e9b8546233cb None None] MFA Rules not processed for user `1bee23d32a3f49b78d485c60dafce049`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:25:17.282740 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-dd30adb3-5f4d-4dd9-b0be-e9b8546233cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:25:17.353183 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 904/1809] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:25:17 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 11:25:17.365239 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd30adb3-5f4d-4dd9-b0be-e9b8546233cb None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:17.367064 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-dd30adb3-5f4d-4dd9-b0be-e9b8546233cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:25:17.377967 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 906/1810] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jun 29 11:25:17 2022] POST /identity/v3/users => generated 294 bytes in 335 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 11:25:17.378976 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec5c109-c5f4-4a7c-9570-061e42416325 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:17.389158 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec5c109-c5f4-4a7c-9570-061e42416325 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:17.426084 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd30adb3-5f4d-4dd9-b0be-e9b8546233cb None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:17.430257 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737efa59-ec68-49fb-8858-c9115ec9bde7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff7931b75a924394b1af733febce0834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:17.430911 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737efa59-ec68-49fb-8858-c9115ec9bde7 service cinder] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:17.431200 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737efa59-ec68-49fb-8858-c9115ec9bde7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:17.431480 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737efa59-ec68-49fb-8858-c9115ec9bde7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:17.441049 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737efa59-ec68-49fb-8858-c9115ec9bde7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:17.442510 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737efa59-ec68-49fb-8858-c9115ec9bde7 service cinder] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:17.445802 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfff357-6e16-44d3-af2c-200f1e180538 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:17.446464 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfff357-6e16-44d3-af2c-200f1e180538 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:17.446746 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfff357-6e16-44d3-af2c-200f1e180538 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:17.447023 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfff357-6e16-44d3-af2c-200f1e180538 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:17.447749 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfff357-6e16-44d3-af2c-200f1e180538 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:17.449430 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfff357-6e16-44d3-af2c-200f1e180538 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:17.452988 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 905/1811] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:25:17 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 11:25:17.459421 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 907/1812] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:25:17 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 11:25:17.463292 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737efa59-ec68-49fb-8858-c9115ec9bde7 service cinder] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:17.474000 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737efa59-ec68-49fb-8858-c9115ec9bde7 service cinder] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:17.478683 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33def2a0-3134-4510-8832-8f2f94e5fe11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:17.479335 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33def2a0-3134-4510-8832-8f2f94e5fe11 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:17.479620 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33def2a0-3134-4510-8832-8f2f94e5fe11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:17.479936 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33def2a0-3134-4510-8832-8f2f94e5fe11 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:17.480662 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33def2a0-3134-4510-8832-8f2f94e5fe11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:17.482241 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33def2a0-3134-4510-8832-8f2f94e5fe11 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:17.495684 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 906/1813] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:25:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 11:25:17.502362 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfff357-6e16-44d3-af2c-200f1e180538 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:17.507029 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfff357-6e16-44d3-af2c-200f1e180538 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:17.512222 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d6ac57-a5df-4ba0-a6be-6a07935529a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:17.513093 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d6ac57-a5df-4ba0-a6be-6a07935529a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:17.513371 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d6ac57-a5df-4ba0-a6be-6a07935529a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:17.513644 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d6ac57-a5df-4ba0-a6be-6a07935529a1 admin admin] PATH_INFO: `/v3/projects/cd38f9511584413fac2861e2c0b8afe5/users/73ec23342fcd4e489cc62bf3233b7dab/roles/e1c88eebe911432cab6a07fbeb84653b` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:17.531958 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d6ac57-a5df-4ba0-a6be-6a07935529a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd38f9511584413fac2861e2c0b8afe5, user_id=73ec23342fcd4e489cc62bf3233b7dab, role_id=e1c88eebe911432cab6a07fbeb84653b)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:17.534859 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d6ac57-a5df-4ba0-a6be-6a07935529a1 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:17.645327 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 908/1814] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jun 29 11:25:17 2022] PUT /identity/v3/projects/cd38f9511584413fac2861e2c0b8afe5/users/73ec23342fcd4e489cc62bf3233b7dab/roles/e1c88eebe911432cab6a07fbeb84653b => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 11:25:17.655619 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebb4f00-36b7-4d7b-a7c3-c1ff4e690834 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:17.655941 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebb4f00-36b7-4d7b-a7c3-c1ff4e690834 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:17.656222 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebb4f00-36b7-4d7b-a7c3-c1ff4e690834 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:17.685033 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-8ebb4f00-36b7-4d7b-a7c3-c1ff4e690834 None None] MFA Rules not processed for user `73ec23342fcd4e489cc62bf3233b7dab`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:25:17.686441 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-8ebb4f00-36b7-4d7b-a7c3-c1ff4e690834 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:25:17.737211 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:25:17.768693 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 907/1815] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:25:17 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:25:17.772962 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d6ac57-a5df-4ba0-a6be-6a07935529a1 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:17.841486 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d6ac57-a5df-4ba0-a6be-6a07935529a1 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:17.849016 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28ed6c1-d2b7-4f47-9594-cb05bee87f37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:17.851524 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28ed6c1-d2b7-4f47-9594-cb05bee87f37 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:17.851882 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28ed6c1-d2b7-4f47-9594-cb05bee87f37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:17.852152 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28ed6c1-d2b7-4f47-9594-cb05bee87f37 admin admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:17.852894 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28ed6c1-d2b7-4f47-9594-cb05bee87f37 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:17.855309 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28ed6c1-d2b7-4f47-9594-cb05bee87f37 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:17.945620 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 909/1816] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jun 29 11:25:17 2022] POST /identity/v3/projects => generated 387 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 11:25:17.946844 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebb4f00-36b7-4d7b-a7c3-c1ff4e690834 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:17.957488 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebb4f00-36b7-4d7b-a7c3-c1ff4e690834 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:18.012952 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffca4bab-9a9c-47f6-8466-321fc3684620 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:18.013635 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffca4bab-9a9c-47f6-8466-321fc3684620 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:18.013923 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffca4bab-9a9c-47f6-8466-321fc3684620 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:18.014199 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffca4bab-9a9c-47f6-8466-321fc3684620 admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:18.014925 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffca4bab-9a9c-47f6-8466-321fc3684620 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:18.016916 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffca4bab-9a9c-47f6-8466-321fc3684620 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:18.114545 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 908/1817] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jun 29 11:25:17 2022] POST /identity/v3/users => generated 300 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:25:18.119607 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28ed6c1-d2b7-4f47-9594-cb05bee87f37 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:18.130831 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28ed6c1-d2b7-4f47-9594-cb05bee87f37 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:18.141565 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d7e1ba-0d67-4860-9b78-54c29aadd8ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:18.142291 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d7e1ba-0d67-4860-9b78-54c29aadd8ff admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:18.142582 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d7e1ba-0d67-4860-9b78-54c29aadd8ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:18.145698 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d7e1ba-0d67-4860-9b78-54c29aadd8ff admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:18.145698 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d7e1ba-0d67-4860-9b78-54c29aadd8ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:18.145698 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d7e1ba-0d67-4860-9b78-54c29aadd8ff admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:18.157395 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 910/1818] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:25:18 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 11:25:18.161635 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffca4bab-9a9c-47f6-8466-321fc3684620 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:18.173122 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffca4bab-9a9c-47f6-8466-321fc3684620 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:18.180527 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1980bf-f90f-4a66-840f-24c62152cafe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:18.181436 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1980bf-f90f-4a66-840f-24c62152cafe admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:18.181736 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1980bf-f90f-4a66-840f-24c62152cafe admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:18.182018 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1980bf-f90f-4a66-840f-24c62152cafe admin admin] PATH_INFO: `/v3/projects/ede3a053ecb54f28ba1129c148f9d6ed/users/2281ceaf0c6b4565952b3ac145d77776/roles/e95c2ce63e114cc19a9dda6ad21e794f` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:18.198774 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1980bf-f90f-4a66-840f-24c62152cafe admin admin] RBAC: Authorizing `identity:create_grant(project_id=ede3a053ecb54f28ba1129c148f9d6ed, user_id=2281ceaf0c6b4565952b3ac145d77776, role_id=e95c2ce63e114cc19a9dda6ad21e794f)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:18.203724 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1980bf-f90f-4a66-840f-24c62152cafe admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:18.275368 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 909/1819] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jun 29 11:25:18 2022] PUT /identity/v3/projects/ede3a053ecb54f28ba1129c148f9d6ed/users/2281ceaf0c6b4565952b3ac145d77776/roles/e95c2ce63e114cc19a9dda6ad21e794f => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:25:18.279302 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d7e1ba-0d67-4860-9b78-54c29aadd8ff admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:18.289921 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d7e1ba-0d67-4860-9b78-54c29aadd8ff admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:18.348710 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e22a80-38c8-47c9-85a5-dac5f3aad22d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:18.350488 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e22a80-38c8-47c9-85a5-dac5f3aad22d admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:18.351408 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e22a80-38c8-47c9-85a5-dac5f3aad22d admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:18.351699 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e22a80-38c8-47c9-85a5-dac5f3aad22d admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:18.352611 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e22a80-38c8-47c9-85a5-dac5f3aad22d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:18.354337 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e22a80-38c8-47c9-85a5-dac5f3aad22d admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:18.368366 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 911/1820] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:25:18 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 11:25:18.369628 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1980bf-f90f-4a66-840f-24c62152cafe admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:18.380053 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1980bf-f90f-4a66-840f-24c62152cafe admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:18.385320 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f67a9a-043f-4621-a769-43c00b9f28fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:18.386206 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f67a9a-043f-4621-a769-43c00b9f28fa admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:18.386504 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f67a9a-043f-4621-a769-43c00b9f28fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:18.386950 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f67a9a-043f-4621-a769-43c00b9f28fa admin admin] PATH_INFO: `/v3/projects/ede3a053ecb54f28ba1129c148f9d6ed/users/2281ceaf0c6b4565952b3ac145d77776/roles/e95c2ce63e114cc19a9dda6ad21e794f` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:18.388962 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f67a9a-043f-4621-a769-43c00b9f28fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=ede3a053ecb54f28ba1129c148f9d6ed, user_id=2281ceaf0c6b4565952b3ac145d77776, role_id=e95c2ce63e114cc19a9dda6ad21e794f)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:18.391576 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f67a9a-043f-4621-a769-43c00b9f28fa admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:18.403248 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 910/1821] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jun 29 11:25:18 2022] PUT /identity/v3/projects/ede3a053ecb54f28ba1129c148f9d6ed/users/2281ceaf0c6b4565952b3ac145d77776/roles/e95c2ce63e114cc19a9dda6ad21e794f => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:25:18.421070 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eff3ced-83ad-4c3a-b588-7532cca0b8bf None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:18.421375 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eff3ced-83ad-4c3a-b588-7532cca0b8bf None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:18.421654 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eff3ced-83ad-4c3a-b588-7532cca0b8bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:18.453224 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-0eff3ced-83ad-4c3a-b588-7532cca0b8bf None None] MFA Rules not processed for user `2281ceaf0c6b4565952b3ac145d77776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:25:18.453224 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-0eff3ced-83ad-4c3a-b588-7532cca0b8bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:25:18.490254 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:25:18.527230 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 912/1822] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:25:18 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:25:18.594248 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f67a9a-043f-4621-a769-43c00b9f28fa admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:18.596628 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eff3ced-83ad-4c3a-b588-7532cca0b8bf None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:18.596886 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-0eff3ced-83ad-4c3a-b588-7532cca0b8bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:25:18.653858 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eff3ced-83ad-4c3a-b588-7532cca0b8bf None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:18.656571 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f67a9a-043f-4621-a769-43c00b9f28fa admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:18.657705 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3613b03-5837-429b-9b37-b81b32fc4d0c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff7931b75a924394b1af733febce0834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:18.658356 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3613b03-5837-429b-9b37-b81b32fc4d0c service cinder] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:18.658642 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3613b03-5837-429b-9b37-b81b32fc4d0c service cinder] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:18.658923 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3613b03-5837-429b-9b37-b81b32fc4d0c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:18.661545 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef5c073-4581-4c2e-aaa3-ec9a6d239c28 tempest-VolumeQuotaClassesTest-2073223389 tempest-VolumeQuotaClassesTest-2073223389-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1bee23d32a3f49b78d485c60dafce049', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e00c5a05fe4c43fa9af22772d290cd20', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:18.662226 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef5c073-4581-4c2e-aaa3-ec9a6d239c28 tempest-VolumeQuotaClassesTest-2073223389 tempest-VolumeQuotaClassesTest-2073223389-project-admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:18.662515 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef5c073-4581-4c2e-aaa3-ec9a6d239c28 tempest-VolumeQuotaClassesTest-2073223389 tempest-VolumeQuotaClassesTest-2073223389-project-admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:18.662804 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef5c073-4581-4c2e-aaa3-ec9a6d239c28 tempest-VolumeQuotaClassesTest-2073223389 tempest-VolumeQuotaClassesTest-2073223389-project-admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:18.663564 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef5c073-4581-4c2e-aaa3-ec9a6d239c28 tempest-VolumeQuotaClassesTest-2073223389 tempest-VolumeQuotaClassesTest-2073223389-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:25:18.664044 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef5c073-4581-4c2e-aaa3-ec9a6d239c28 tempest-VolumeQuotaClassesTest-2073223389 tempest-VolumeQuotaClassesTest-2073223389-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:18.665644 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef5c073-4581-4c2e-aaa3-ec9a6d239c28 tempest-VolumeQuotaClassesTest-2073223389 tempest-VolumeQuotaClassesTest-2073223389-project-admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:18.668956 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3613b03-5837-429b-9b37-b81b32fc4d0c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:18.670560 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3613b03-5837-429b-9b37-b81b32fc4d0c service cinder] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:18.679593 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 911/1823] 10.222.0.44 () {58 vars in 1152 bytes} [Wed Jun 29 11:25:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:25:18.683108 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 913/1824] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:25:18 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 11:25:18.685695 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef5c073-4581-4c2e-aaa3-ec9a6d239c28 tempest-VolumeQuotaClassesTest-2073223389 tempest-VolumeQuotaClassesTest-2073223389-project-admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:18.698217 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef5c073-4581-4c2e-aaa3-ec9a6d239c28 tempest-VolumeQuotaClassesTest-2073223389 tempest-VolumeQuotaClassesTest-2073223389-project-admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:18.705986 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f60b18-0520-426d-aed2-559931ca7c97 tempest-VolumeQuotaClassesTest-2073223389 tempest-VolumeQuotaClassesTest-2073223389-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1bee23d32a3f49b78d485c60dafce049', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e00c5a05fe4c43fa9af22772d290cd20', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:18.706663 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f60b18-0520-426d-aed2-559931ca7c97 tempest-VolumeQuotaClassesTest-2073223389 tempest-VolumeQuotaClassesTest-2073223389-project-admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:18.706959 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f60b18-0520-426d-aed2-559931ca7c97 tempest-VolumeQuotaClassesTest-2073223389 tempest-VolumeQuotaClassesTest-2073223389-project-admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:18.707249 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f60b18-0520-426d-aed2-559931ca7c97 tempest-VolumeQuotaClassesTest-2073223389 tempest-VolumeQuotaClassesTest-2073223389-project-admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:18.708010 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f60b18-0520-426d-aed2-559931ca7c97 tempest-VolumeQuotaClassesTest-2073223389 tempest-VolumeQuotaClassesTest-2073223389-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:18.709890 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f60b18-0520-426d-aed2-559931ca7c97 tempest-VolumeQuotaClassesTest-2073223389 tempest-VolumeQuotaClassesTest-2073223389-project-admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:18.817504 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 912/1825] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jun 29 11:25:18 2022] POST /identity/v3/projects => generated 353 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 11:25:18.823019 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3613b03-5837-429b-9b37-b81b32fc4d0c service cinder] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:18.841803 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3613b03-5837-429b-9b37-b81b32fc4d0c service cinder] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:19.001136 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056827ee-cdd4-4dd2-ad09-410913f69513 tempest-VolumeQuotaClassesTest-2073223389 tempest-VolumeQuotaClassesTest-2073223389-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1bee23d32a3f49b78d485c60dafce049', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e00c5a05fe4c43fa9af22772d290cd20', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:19.005162 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056827ee-cdd4-4dd2-ad09-410913f69513 tempest-VolumeQuotaClassesTest-2073223389 tempest-VolumeQuotaClassesTest-2073223389-project-admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:19.005162 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056827ee-cdd4-4dd2-ad09-410913f69513 tempest-VolumeQuotaClassesTest-2073223389 tempest-VolumeQuotaClassesTest-2073223389-project-admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:19.005162 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056827ee-cdd4-4dd2-ad09-410913f69513 tempest-VolumeQuotaClassesTest-2073223389 tempest-VolumeQuotaClassesTest-2073223389-project-admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:19.005625 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056827ee-cdd4-4dd2-ad09-410913f69513 tempest-VolumeQuotaClassesTest-2073223389 tempest-VolumeQuotaClassesTest-2073223389-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:25:19.006027 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056827ee-cdd4-4dd2-ad09-410913f69513 tempest-VolumeQuotaClassesTest-2073223389 tempest-VolumeQuotaClassesTest-2073223389-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:19.008521 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056827ee-cdd4-4dd2-ad09-410913f69513 tempest-VolumeQuotaClassesTest-2073223389 tempest-VolumeQuotaClassesTest-2073223389-project-admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:19.024748 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 914/1826] 10.222.0.44 () {58 vars in 1152 bytes} [Wed Jun 29 11:25:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:25:19.083398 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f60b18-0520-426d-aed2-559931ca7c97 tempest-VolumeQuotaClassesTest-2073223389 tempest-VolumeQuotaClassesTest-2073223389-project-admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:19.095291 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f60b18-0520-426d-aed2-559931ca7c97 tempest-VolumeQuotaClassesTest-2073223389 tempest-VolumeQuotaClassesTest-2073223389-project-admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:19.100484 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7386172b-f431-4eba-9c7e-df31d35a8108 tempest-VolumeQuotaClassesTest-2073223389 tempest-VolumeQuotaClassesTest-2073223389-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1bee23d32a3f49b78d485c60dafce049', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e00c5a05fe4c43fa9af22772d290cd20', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:19.102037 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7386172b-f431-4eba-9c7e-df31d35a8108 tempest-VolumeQuotaClassesTest-2073223389 tempest-VolumeQuotaClassesTest-2073223389-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:19.102400 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7386172b-f431-4eba-9c7e-df31d35a8108 tempest-VolumeQuotaClassesTest-2073223389 tempest-VolumeQuotaClassesTest-2073223389-project-admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:19.102771 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7386172b-f431-4eba-9c7e-df31d35a8108 tempest-VolumeQuotaClassesTest-2073223389 tempest-VolumeQuotaClassesTest-2073223389-project-admin] PATH_INFO: `/v3/projects/358b27e1a9944052b1f7300ef945de93` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:19.104650 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7386172b-f431-4eba-9c7e-df31d35a8108 tempest-VolumeQuotaClassesTest-2073223389 tempest-VolumeQuotaClassesTest-2073223389-project-admin] RBAC: Authorizing `identity:delete_project(project_id=358b27e1a9944052b1f7300ef945de93)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:19.106676 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7386172b-f431-4eba-9c7e-df31d35a8108 tempest-VolumeQuotaClassesTest-2073223389 tempest-VolumeQuotaClassesTest-2073223389-project-admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:19.143543 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:25:19.212462 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:25:19.233599 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-7386172b-f431-4eba-9c7e-df31d35a8108 tempest-VolumeQuotaClassesTest-2073223389 tempest-VolumeQuotaClassesTest-2073223389-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '358b27e1a9944052b1f7300ef945de93', 'request_id': 'req-7386172b-f431-4eba-9c7e-df31d35a8108'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:19.240172 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-7386172b-f431-4eba-9c7e-df31d35a8108 tempest-VolumeQuotaClassesTest-2073223389 tempest-VolumeQuotaClassesTest-2073223389-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '358b27e1a9944052b1f7300ef945de93', 'request_id': 'req-7386172b-f431-4eba-9c7e-df31d35a8108'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:19.248107 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-7386172b-f431-4eba-9c7e-df31d35a8108 tempest-VolumeQuotaClassesTest-2073223389 tempest-VolumeQuotaClassesTest-2073223389-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '358b27e1a9944052b1f7300ef945de93', 'request_id': 'req-7386172b-f431-4eba-9c7e-df31d35a8108'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:19.287245 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8717677-33a8-491b-ac4b-fd78255cf2d8 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:19.287788 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8717677-33a8-491b-ac4b-fd78255cf2d8 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:19.289042 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8717677-33a8-491b-ac4b-fd78255cf2d8 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:19.291232 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 915/1827] 10.222.0.44 () {54 vars in 921 bytes} [Wed Jun 29 11:25:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 29 11:25:19.323676 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38c1c9f-09e2-481d-98a6-9962a018d8f5 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:19.324028 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38c1c9f-09e2-481d-98a6-9962a018d8f5 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:19.324513 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38c1c9f-09e2-481d-98a6-9962a018d8f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:19.345553 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-b38c1c9f-09e2-481d-98a6-9962a018d8f5 None None] MFA Rules not processed for user `796aeeff16df4739bddec1f6dee61bfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:25:19.346826 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-b38c1c9f-09e2-481d-98a6-9962a018d8f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:25:19.360906 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-7386172b-f431-4eba-9c7e-df31d35a8108 tempest-VolumeQuotaClassesTest-2073223389 tempest-VolumeQuotaClassesTest-2073223389-project-admin] The token cache is being invalidate because project 358b27e1a9944052b1f7300ef945de93 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 11:25:19.361876 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-7386172b-f431-4eba-9c7e-df31d35a8108 tempest-VolumeQuotaClassesTest-2073223389 tempest-VolumeQuotaClassesTest-2073223389-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:19.363108 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-7386172b-f431-4eba-9c7e-df31d35a8108 tempest-VolumeQuotaClassesTest-2073223389 tempest-VolumeQuotaClassesTest-2073223389-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:19.365071 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 913/1828] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jun 29 11:25:19 2022] DELETE /identity/v3/projects/358b27e1a9944052b1f7300ef945de93 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:25:19.427304 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 916/1829] 10.222.0.44 () {58 vars in 1018 bytes} [Wed Jun 29 11:25:19 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 11:25:19.451188 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7386172b-f431-4eba-9c7e-df31d35a8108 tempest-VolumeQuotaClassesTest-2073223389 tempest-VolumeQuotaClassesTest-2073223389-project-admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:19.453180 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-7386172b-f431-4eba-9c7e-df31d35a8108 tempest-VolumeQuotaClassesTest-2073223389 tempest-VolumeQuotaClassesTest-2073223389-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:25:19.481306 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7386172b-f431-4eba-9c7e-df31d35a8108 tempest-VolumeQuotaClassesTest-2073223389 tempest-VolumeQuotaClassesTest-2073223389-project-admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:19.490105 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbbb06f-289d-47cb-87e8-79ae5db964f0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '796aeeff16df4739bddec1f6dee61bfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:19.491063 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbbb06f-289d-47cb-87e8-79ae5db964f0 service nova] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:19.491590 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbbb06f-289d-47cb-87e8-79ae5db964f0 service nova] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:19.492051 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbbb06f-289d-47cb-87e8-79ae5db964f0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:19.503450 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbbb06f-289d-47cb-87e8-79ae5db964f0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:19.505232 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbbb06f-289d-47cb-87e8-79ae5db964f0 service nova] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:19.516455 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 914/1830] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:25:19 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:25:20.226757 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38c1c9f-09e2-481d-98a6-9962a018d8f5 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:20.228273 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-b38c1c9f-09e2-481d-98a6-9962a018d8f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:25:20.242524 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:25:20.262138 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbbb06f-289d-47cb-87e8-79ae5db964f0 service nova] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:20.263753 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-7bbbb06f-289d-47cb-87e8-79ae5db964f0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:25:20.333376 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:25:20.351300 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38c1c9f-09e2-481d-98a6-9962a018d8f5 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:20.361346 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899762ad-bd11-48ca-9fc2-cd255646c5b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:20.362400 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899762ad-bd11-48ca-9fc2-cd255646c5b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:20.363051 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899762ad-bd11-48ca-9fc2-cd255646c5b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:20.363343 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899762ad-bd11-48ca-9fc2-cd255646c5b1 admin admin] PATH_INFO: `/v3/users/ca20ad93e1394f129531756c7bbc2f98` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:20.366980 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899762ad-bd11-48ca-9fc2-cd255646c5b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca20ad93e1394f129531756c7bbc2f98)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:20.370012 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899762ad-bd11-48ca-9fc2-cd255646c5b1 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:20.384714 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbbb06f-289d-47cb-87e8-79ae5db964f0 service nova] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:20.390638 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e668e7b0-72bd-4e9c-a804-26d46e911524 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:20.391528 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e668e7b0-72bd-4e9c-a804-26d46e911524 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:20.391851 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e668e7b0-72bd-4e9c-a804-26d46e911524 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:20.392108 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e668e7b0-72bd-4e9c-a804-26d46e911524 admin admin] PATH_INFO: `/v3/users/979606c20ef3442e808b191ce497df58` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:20.393406 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e668e7b0-72bd-4e9c-a804-26d46e911524 admin admin] RBAC: Authorizing `identity:delete_user(user_id=979606c20ef3442e808b191ce497df58)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:20.395468 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e668e7b0-72bd-4e9c-a804-26d46e911524 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:20.671855 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-899762ad-bd11-48ca-9fc2-cd255646c5b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca20ad93e1394f129531756c7bbc2f98'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:20.718781 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-e668e7b0-72bd-4e9c-a804-26d46e911524 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '979606c20ef3442e808b191ce497df58'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:20.719872 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-e668e7b0-72bd-4e9c-a804-26d46e911524 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '979606c20ef3442e808b191ce497df58'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:20.775450 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-899762ad-bd11-48ca-9fc2-cd255646c5b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca20ad93e1394f129531756c7bbc2f98'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:20.776181 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-899762ad-bd11-48ca-9fc2-cd255646c5b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca20ad93e1394f129531756c7bbc2f98'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:20.788576 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-899762ad-bd11-48ca-9fc2-cd255646c5b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca20ad93e1394f129531756c7bbc2f98'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:20.789247 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-899762ad-bd11-48ca-9fc2-cd255646c5b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca20ad93e1394f129531756c7bbc2f98'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:20.799956 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:25:20.821716 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 917/1831] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jun 29 11:25:20 2022] DELETE /identity/v3/users/ca20ad93e1394f129531756c7bbc2f98 => generated 0 bytes in 592 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:25:20.824480 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd1ee77-7b21-46b5-8bfc-bb7df3c390d3 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:20.824793 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd1ee77-7b21-46b5-8bfc-bb7df3c390d3 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:20.825080 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd1ee77-7b21-46b5-8bfc-bb7df3c390d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:20.850954 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-7cd1ee77-7b21-46b5-8bfc-bb7df3c390d3 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:25:20.852262 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-7cd1ee77-7b21-46b5-8bfc-bb7df3c390d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:25:20.856583 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-e668e7b0-72bd-4e9c-a804-26d46e911524 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '979606c20ef3442e808b191ce497df58'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:20.868284 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-e668e7b0-72bd-4e9c-a804-26d46e911524 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '979606c20ef3442e808b191ce497df58'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:20.870035 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-e668e7b0-72bd-4e9c-a804-26d46e911524 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '979606c20ef3442e808b191ce497df58'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:20.899872 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 915/1832] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jun 29 11:25:20 2022] DELETE /identity/v3/users/979606c20ef3442e808b191ce497df58 => generated 0 bytes in 637 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 11:25:20.906678 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8926c2d0-faf3-472c-a116-d17f032145c5 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:20.906973 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8926c2d0-faf3-472c-a116-d17f032145c5 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:20.907418 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8926c2d0-faf3-472c-a116-d17f032145c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:20.946724 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 918/1833] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:25:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 11:25:20.953859 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-8926c2d0-faf3-472c-a116-d17f032145c5 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:25:20.956185 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-8926c2d0-faf3-472c-a116-d17f032145c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:25:20.957083 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd1ee77-7b21-46b5-8bfc-bb7df3c390d3 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:20.961153 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-7cd1ee77-7b21-46b5-8bfc-bb7df3c390d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:25:21.048475 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 916/1834] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:25:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:25:21.052518 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd1ee77-7b21-46b5-8bfc-bb7df3c390d3 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:21.061729 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8926c2d0-faf3-472c-a116-d17f032145c5 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:21.098970 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:25:21.146566 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8926c2d0-faf3-472c-a116-d17f032145c5 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:21.153194 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8677755d-bb71-4c80-bb8d-e518fc2b266e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f4bb9f989e4024a5da28dced646b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:21.154724 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8677755d-bb71-4c80-bb8d-e518fc2b266e service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:21.154724 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8677755d-bb71-4c80-bb8d-e518fc2b266e service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:21.154724 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8677755d-bb71-4c80-bb8d-e518fc2b266e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:21.157540 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4daba8b-65a0-4b60-af72-30cb0c0e2634 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f4bb9f989e4024a5da28dced646b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:21.158651 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4daba8b-65a0-4b60-af72-30cb0c0e2634 service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:21.160325 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4daba8b-65a0-4b60-af72-30cb0c0e2634 service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:21.160811 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4daba8b-65a0-4b60-af72-30cb0c0e2634 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:21.172415 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8677755d-bb71-4c80-bb8d-e518fc2b266e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:21.174063 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4daba8b-65a0-4b60-af72-30cb0c0e2634 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:21.174430 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8677755d-bb71-4c80-bb8d-e518fc2b266e service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:21.176086 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4daba8b-65a0-4b60-af72-30cb0c0e2634 service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:21.189179 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 919/1836] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:25:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 11:25:21.189402 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 917/1836] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:25:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 11:25:21.434694 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:25:21.587372 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4daba8b-65a0-4b60-af72-30cb0c0e2634 service neutron] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:21.588852 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-a4daba8b-65a0-4b60-af72-30cb0c0e2634 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:25:21.600754 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:25:21.653482 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4daba8b-65a0-4b60-af72-30cb0c0e2634 service neutron] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:21.657291 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933f5271-5ebb-469f-b9a3-ceb25f709557 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff7931b75a924394b1af733febce0834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:21.657953 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933f5271-5ebb-469f-b9a3-ceb25f709557 service cinder] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:21.658371 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933f5271-5ebb-469f-b9a3-ceb25f709557 service cinder] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:21.659277 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933f5271-5ebb-469f-b9a3-ceb25f709557 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:21.661727 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-933f5271-5ebb-469f-b9a3-ceb25f709557 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:25:21.701525 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:25:21.745422 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933f5271-5ebb-469f-b9a3-ceb25f709557 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:21.746892 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933f5271-5ebb-469f-b9a3-ceb25f709557 service cinder] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:21.758055 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 920/1837] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:25:21 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:25:21.817934 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:25:21.961932 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8677755d-bb71-4c80-bb8d-e518fc2b266e service neutron] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:21.963601 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-8677755d-bb71-4c80-bb8d-e518fc2b266e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:25:21.998818 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8677755d-bb71-4c80-bb8d-e518fc2b266e service neutron] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:22.010247 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa32d49-7344-4deb-a019-e52a5c70679b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:22.011208 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa32d49-7344-4deb-a019-e52a5c70679b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:22.011578 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa32d49-7344-4deb-a019-e52a5c70679b admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:22.011962 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa32d49-7344-4deb-a019-e52a5c70679b admin admin] PATH_INFO: `/v3/projects/7cb53b7fa6cd47358735e8ba3907e8b6` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:22.013404 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa32d49-7344-4deb-a019-e52a5c70679b admin admin] RBAC: Authorizing `identity:delete_project(project_id=7cb53b7fa6cd47358735e8ba3907e8b6)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:22.015628 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa32d49-7344-4deb-a019-e52a5c70679b admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:22.160147 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-7aa32d49-7344-4deb-a019-e52a5c70679b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7cb53b7fa6cd47358735e8ba3907e8b6', 'request_id': 'req-7aa32d49-7344-4deb-a019-e52a5c70679b'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:22.167563 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-7aa32d49-7344-4deb-a019-e52a5c70679b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7cb53b7fa6cd47358735e8ba3907e8b6', 'request_id': 'req-7aa32d49-7344-4deb-a019-e52a5c70679b'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:22.175206 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-7aa32d49-7344-4deb-a019-e52a5c70679b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7cb53b7fa6cd47358735e8ba3907e8b6', 'request_id': 'req-7aa32d49-7344-4deb-a019-e52a5c70679b'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:22.238634 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933f5271-5ebb-469f-b9a3-ceb25f709557 service cinder] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:22.240159 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-933f5271-5ebb-469f-b9a3-ceb25f709557 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:25:22.254892 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933f5271-5ebb-469f-b9a3-ceb25f709557 service cinder] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:22.291975 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-7aa32d49-7344-4deb-a019-e52a5c70679b admin admin] The token cache is being invalidate because project 7cb53b7fa6cd47358735e8ba3907e8b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 11:25:22.292780 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-7aa32d49-7344-4deb-a019-e52a5c70679b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:22.295367 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-7aa32d49-7344-4deb-a019-e52a5c70679b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:22.298681 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 918/1838] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jun 29 11:25:21 2022] DELETE /identity/v3/projects/7cb53b7fa6cd47358735e8ba3907e8b6 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:25:22.304246 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa32d49-7344-4deb-a019-e52a5c70679b admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:22.308245 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-7aa32d49-7344-4deb-a019-e52a5c70679b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:25:22.326749 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad71c6fd-ef1e-465c-8852-55c5631f9941 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:22.327735 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad71c6fd-ef1e-465c-8852-55c5631f9941 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:22.328407 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad71c6fd-ef1e-465c-8852-55c5631f9941 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:22.328663 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad71c6fd-ef1e-465c-8852-55c5631f9941 admin admin] PATH_INFO: `/v3/projects/42a37aad69cf49708fdb70896cb09ba1` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:22.329972 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad71c6fd-ef1e-465c-8852-55c5631f9941 admin admin] RBAC: Authorizing `identity:delete_project(project_id=42a37aad69cf49708fdb70896cb09ba1)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:22.332448 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad71c6fd-ef1e-465c-8852-55c5631f9941 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:22.389390 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa32d49-7344-4deb-a019-e52a5c70679b admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:22.404290 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83921183-7779-45f0-b631-9cd1c6720fa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:22.404290 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83921183-7779-45f0-b631-9cd1c6720fa0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:22.404290 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83921183-7779-45f0-b631-9cd1c6720fa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:22.404290 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83921183-7779-45f0-b631-9cd1c6720fa0 admin admin] PATH_INFO: `/v3/users/1bee23d32a3f49b78d485c60dafce049` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:22.404687 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83921183-7779-45f0-b631-9cd1c6720fa0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1bee23d32a3f49b78d485c60dafce049)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:22.404687 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83921183-7779-45f0-b631-9cd1c6720fa0 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:22.442876 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-ad71c6fd-ef1e-465c-8852-55c5631f9941 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '42a37aad69cf49708fdb70896cb09ba1', 'request_id': 'req-ad71c6fd-ef1e-465c-8852-55c5631f9941'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:22.537577 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-ad71c6fd-ef1e-465c-8852-55c5631f9941 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '42a37aad69cf49708fdb70896cb09ba1', 'request_id': 'req-ad71c6fd-ef1e-465c-8852-55c5631f9941'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:22.545421 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-ad71c6fd-ef1e-465c-8852-55c5631f9941 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '42a37aad69cf49708fdb70896cb09ba1', 'request_id': 'req-ad71c6fd-ef1e-465c-8852-55c5631f9941'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:22.556965 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-ad71c6fd-ef1e-465c-8852-55c5631f9941 admin admin] The token cache is being invalidate because project 42a37aad69cf49708fdb70896cb09ba1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 11:25:22.557351 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-ad71c6fd-ef1e-465c-8852-55c5631f9941 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:22.558035 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-ad71c6fd-ef1e-465c-8852-55c5631f9941 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:22.563235 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 921/1839] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jun 29 11:25:22 2022] DELETE /identity/v3/projects/42a37aad69cf49708fdb70896cb09ba1 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 11:25:22.564167 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad71c6fd-ef1e-465c-8852-55c5631f9941 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:22.565508 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-ad71c6fd-ef1e-465c-8852-55c5631f9941 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:25:22.628830 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad71c6fd-ef1e-465c-8852-55c5631f9941 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:22.633521 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dfec57-3cb1-4087-b602-0d0a4fcebcd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:22.634386 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dfec57-3cb1-4087-b602-0d0a4fcebcd6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:22.634668 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dfec57-3cb1-4087-b602-0d0a4fcebcd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:22.634943 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dfec57-3cb1-4087-b602-0d0a4fcebcd6 admin admin] PATH_INFO: `/v3/users/4b960532407841e1886eae53db74cc96` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:22.636152 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35dfec57-3cb1-4087-b602-0d0a4fcebcd6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b960532407841e1886eae53db74cc96)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:22.638064 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35dfec57-3cb1-4087-b602-0d0a4fcebcd6 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:22.713041 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-83921183-7779-45f0-b631-9cd1c6720fa0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1bee23d32a3f49b78d485c60dafce049'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:22.713917 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-83921183-7779-45f0-b631-9cd1c6720fa0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1bee23d32a3f49b78d485c60dafce049'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:22.799869 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-83921183-7779-45f0-b631-9cd1c6720fa0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1bee23d32a3f49b78d485c60dafce049'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:22.808156 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-83921183-7779-45f0-b631-9cd1c6720fa0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1bee23d32a3f49b78d485c60dafce049'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:22.808829 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-83921183-7779-45f0-b631-9cd1c6720fa0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1bee23d32a3f49b78d485c60dafce049'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:22.826208 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 919/1840] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jun 29 11:25:22 2022] DELETE /identity/v3/users/1bee23d32a3f49b78d485c60dafce049 => generated 0 bytes in 524 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:25:22.828733 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:25:22.903807 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-35dfec57-3cb1-4087-b602-0d0a4fcebcd6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b960532407841e1886eae53db74cc96'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:22.933194 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:25:22.971793 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-35dfec57-3cb1-4087-b602-0d0a4fcebcd6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b960532407841e1886eae53db74cc96'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:22.973400 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-35dfec57-3cb1-4087-b602-0d0a4fcebcd6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b960532407841e1886eae53db74cc96'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:22.986696 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-35dfec57-3cb1-4087-b602-0d0a4fcebcd6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b960532407841e1886eae53db74cc96'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:22.987489 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-35dfec57-3cb1-4087-b602-0d0a4fcebcd6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b960532407841e1886eae53db74cc96'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:23.001657 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 922/1841] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jun 29 11:25:22 2022] DELETE /identity/v3/users/4b960532407841e1886eae53db74cc96 => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 11:25:23.476171 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dfec57-3cb1-4087-b602-0d0a4fcebcd6 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:23.477073 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83921183-7779-45f0-b631-9cd1c6720fa0 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:23.477597 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-35dfec57-3cb1-4087-b602-0d0a4fcebcd6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:25:23.477713 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-83921183-7779-45f0-b631-9cd1c6720fa0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:25:23.562003 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dfec57-3cb1-4087-b602-0d0a4fcebcd6 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:23.566707 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c160415-dcd7-481d-8181-78f3a9f3b140 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:23.566879 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83921183-7779-45f0-b631-9cd1c6720fa0 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:23.567579 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c160415-dcd7-481d-8181-78f3a9f3b140 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:23.567903 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c160415-dcd7-481d-8181-78f3a9f3b140 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:23.568193 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c160415-dcd7-481d-8181-78f3a9f3b140 admin admin] PATH_INFO: `/v3/projects/381814a417d7459c8a3fe9b92991c46f` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:23.569455 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c160415-dcd7-481d-8181-78f3a9f3b140 admin admin] RBAC: Authorizing `identity:delete_project(project_id=381814a417d7459c8a3fe9b92991c46f)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:23.571442 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4feffd-fe30-4841-8258-9594e59bed28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:23.571569 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c160415-dcd7-481d-8181-78f3a9f3b140 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:23.572341 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4feffd-fe30-4841-8258-9594e59bed28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:23.572621 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4feffd-fe30-4841-8258-9594e59bed28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:23.572897 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4feffd-fe30-4841-8258-9594e59bed28 admin admin] PATH_INFO: `/v3/projects/e00c5a05fe4c43fa9af22772d290cd20` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:23.574075 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4feffd-fe30-4841-8258-9594e59bed28 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e00c5a05fe4c43fa9af22772d290cd20)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:23.576065 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4feffd-fe30-4841-8258-9594e59bed28 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:23.702789 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-3c160415-dcd7-481d-8181-78f3a9f3b140 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '381814a417d7459c8a3fe9b92991c46f', 'request_id': 'req-3c160415-dcd7-481d-8181-78f3a9f3b140'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:23.704013 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-1c4feffd-fe30-4841-8258-9594e59bed28 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e00c5a05fe4c43fa9af22772d290cd20', 'request_id': 'req-1c4feffd-fe30-4841-8258-9594e59bed28'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:23.710488 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-1c4feffd-fe30-4841-8258-9594e59bed28 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e00c5a05fe4c43fa9af22772d290cd20', 'request_id': 'req-1c4feffd-fe30-4841-8258-9594e59bed28'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:23.716219 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-1c4feffd-fe30-4841-8258-9594e59bed28 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e00c5a05fe4c43fa9af22772d290cd20', 'request_id': 'req-1c4feffd-fe30-4841-8258-9594e59bed28'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:23.788546 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-3c160415-dcd7-481d-8181-78f3a9f3b140 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '381814a417d7459c8a3fe9b92991c46f', 'request_id': 'req-3c160415-dcd7-481d-8181-78f3a9f3b140'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:23.796183 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-3c160415-dcd7-481d-8181-78f3a9f3b140 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '381814a417d7459c8a3fe9b92991c46f', 'request_id': 'req-3c160415-dcd7-481d-8181-78f3a9f3b140'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:23.809787 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-3c160415-dcd7-481d-8181-78f3a9f3b140 admin admin] The token cache is being invalidate because project 381814a417d7459c8a3fe9b92991c46f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 11:25:23.810140 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-3c160415-dcd7-481d-8181-78f3a9f3b140 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:23.810813 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-3c160415-dcd7-481d-8181-78f3a9f3b140 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:23.812447 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 923/1842] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jun 29 11:25:23 2022] DELETE /identity/v3/projects/381814a417d7459c8a3fe9b92991c46f => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:25:23.816858 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c160415-dcd7-481d-8181-78f3a9f3b140 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:23.818193 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-3c160415-dcd7-481d-8181-78f3a9f3b140 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:25:23.865090 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:25:23.881419 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-1c4feffd-fe30-4841-8258-9594e59bed28 admin admin] The token cache is being invalidate because project e00c5a05fe4c43fa9af22772d290cd20 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 11:25:23.881756 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-1c4feffd-fe30-4841-8258-9594e59bed28 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:23.882789 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-1c4feffd-fe30-4841-8258-9594e59bed28 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:23.884858 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 920/1843] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jun 29 11:25:23 2022] DELETE /identity/v3/projects/e00c5a05fe4c43fa9af22772d290cd20 => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:25:23.895805 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bcced8-51e2-4722-87be-76031a7e5f1e None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:23.896140 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bcced8-51e2-4722-87be-76031a7e5f1e None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:23.896429 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bcced8-51e2-4722-87be-76031a7e5f1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:23.905117 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c160415-dcd7-481d-8181-78f3a9f3b140 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:23.961211 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-18bcced8-51e2-4722-87be-76031a7e5f1e None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:25:23.963347 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-18bcced8-51e2-4722-87be-76031a7e5f1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:25:23.984610 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba58465-325a-436c-89da-f69ee5d953c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:23.987183 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba58465-325a-436c-89da-f69ee5d953c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:23.987183 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba58465-325a-436c-89da-f69ee5d953c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:23.987183 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba58465-325a-436c-89da-f69ee5d953c5 admin admin] PATH_INFO: `/v3/users/f89a284f25484d1b92f4003091a5638d` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:23.987506 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba58465-325a-436c-89da-f69ee5d953c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f89a284f25484d1b92f4003091a5638d)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:23.989559 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba58465-325a-436c-89da-f69ee5d953c5 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:24.041710 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 921/1844] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:25:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:25:24.043176 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6bff80-f06a-49b2-b40b-f3091b1ad453 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:24.043543 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6bff80-f06a-49b2-b40b-f3091b1ad453 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:24.043786 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6bff80-f06a-49b2-b40b-f3091b1ad453 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:24.061979 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-2d6bff80-f06a-49b2-b40b-f3091b1ad453 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:25:24.063184 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-2d6bff80-f06a-49b2-b40b-f3091b1ad453 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:25:24.071038 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 922/1845] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:25:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:25:24.072441 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bbb879-4054-4367-b167-2c810eaddf10 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:24.072726 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bbb879-4054-4367-b167-2c810eaddf10 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:24.073013 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bbb879-4054-4367-b167-2c810eaddf10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:24.090863 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-47bbb879-4054-4367-b167-2c810eaddf10 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:25:24.092068 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-47bbb879-4054-4367-b167-2c810eaddf10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:25:24.099941 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 923/1846] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:25:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:25:24.101294 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6189d02e-75fa-4538-84e7-13a31abcbac2 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:24.101575 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6189d02e-75fa-4538-84e7-13a31abcbac2 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:24.102023 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6189d02e-75fa-4538-84e7-13a31abcbac2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:24.119683 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-6189d02e-75fa-4538-84e7-13a31abcbac2 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:25:24.120927 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-6189d02e-75fa-4538-84e7-13a31abcbac2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:25:24.128762 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 924/1847] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:25:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:25:24.130924 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6189d02e-75fa-4538-84e7-13a31abcbac2 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:24.138664 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6189d02e-75fa-4538-84e7-13a31abcbac2 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:24.143487 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbee1a6-1a56-4dc3-84b4-ded3ebfbfa94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:24.144051 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbee1a6-1a56-4dc3-84b4-ded3ebfbfa94 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:24.144333 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbee1a6-1a56-4dc3-84b4-ded3ebfbfa94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:24.144608 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbee1a6-1a56-4dc3-84b4-ded3ebfbfa94 admin admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:24.145343 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbee1a6-1a56-4dc3-84b4-ded3ebfbfa94 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:25:24.145649 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbee1a6-1a56-4dc3-84b4-ded3ebfbfa94 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:24.147223 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbee1a6-1a56-4dc3-84b4-ded3ebfbfa94 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:24.161131 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 925/1848] 10.222.0.44 () {58 vars in 1152 bytes} [Wed Jun 29 11:25:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:25:24.162009 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbee1a6-1a56-4dc3-84b4-ded3ebfbfa94 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:24.172401 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbee1a6-1a56-4dc3-84b4-ded3ebfbfa94 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:24.174258 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:25:24.180213 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cfb402-8690-4b54-a24a-9239b358ef2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:24.180883 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cfb402-8690-4b54-a24a-9239b358ef2d admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:24.183902 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cfb402-8690-4b54-a24a-9239b358ef2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:24.183902 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cfb402-8690-4b54-a24a-9239b358ef2d admin admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:24.184075 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-4ba58465-325a-436c-89da-f69ee5d953c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f89a284f25484d1b92f4003091a5638d'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:24.184189 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cfb402-8690-4b54-a24a-9239b358ef2d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:25:24.184189 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cfb402-8690-4b54-a24a-9239b358ef2d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:24.184369 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cfb402-8690-4b54-a24a-9239b358ef2d admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:24.197032 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 926/1849] 10.222.0.44 () {58 vars in 1152 bytes} [Wed Jun 29 11:25:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:25:24.197032 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cfb402-8690-4b54-a24a-9239b358ef2d admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:24.208646 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cfb402-8690-4b54-a24a-9239b358ef2d admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:24.213967 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68eeb90c-123b-400e-8b0a-c8b9c59a86b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:24.215053 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68eeb90c-123b-400e-8b0a-c8b9c59a86b2 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:24.215489 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68eeb90c-123b-400e-8b0a-c8b9c59a86b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:24.215949 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68eeb90c-123b-400e-8b0a-c8b9c59a86b2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:24.217004 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68eeb90c-123b-400e-8b0a-c8b9c59a86b2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:24.220389 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68eeb90c-123b-400e-8b0a-c8b9c59a86b2 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:24.248879 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-4ba58465-325a-436c-89da-f69ee5d953c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f89a284f25484d1b92f4003091a5638d'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:24.249547 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-4ba58465-325a-436c-89da-f69ee5d953c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f89a284f25484d1b92f4003091a5638d'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:24.262422 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-4ba58465-325a-436c-89da-f69ee5d953c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f89a284f25484d1b92f4003091a5638d'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:24.263142 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-4ba58465-325a-436c-89da-f69ee5d953c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f89a284f25484d1b92f4003091a5638d'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:24.276117 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 924/1850] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jun 29 11:25:23 2022] DELETE /identity/v3/users/f89a284f25484d1b92f4003091a5638d => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:25:24.277173 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba58465-325a-436c-89da-f69ee5d953c5 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:24.278510 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-4ba58465-325a-436c-89da-f69ee5d953c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:25:24.334431 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 927/1851] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jun 29 11:25:24 2022] POST /identity/v3/projects => generated 385 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:25:24.334431 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386bf73a-4bab-4069-a14c-ae03c2e0550c None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:24.334431 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386bf73a-4bab-4069-a14c-ae03c2e0550c None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:24.334751 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386bf73a-4bab-4069-a14c-ae03c2e0550c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:24.351117 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba58465-325a-436c-89da-f69ee5d953c5 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:24.353283 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-386bf73a-4bab-4069-a14c-ae03c2e0550c None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:25:24.354498 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-386bf73a-4bab-4069-a14c-ae03c2e0550c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:25:24.422457 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 928/1852] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:25:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:25:24.423380 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386bf73a-4bab-4069-a14c-ae03c2e0550c None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:24.424765 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-386bf73a-4bab-4069-a14c-ae03c2e0550c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:25:24.438007 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386bf73a-4bab-4069-a14c-ae03c2e0550c None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:24.439348 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd3d070-af9f-4e41-a631-172e85342881 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:24.440284 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd3d070-af9f-4e41-a631-172e85342881 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:24.440579 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd3d070-af9f-4e41-a631-172e85342881 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:24.440866 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd3d070-af9f-4e41-a631-172e85342881 admin admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:24.441643 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd3d070-af9f-4e41-a631-172e85342881 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:24.443320 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe52864-1317-4ca5-95bb-87cbe80e81f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:24.443524 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd3d070-af9f-4e41-a631-172e85342881 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:24.444191 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe52864-1317-4ca5-95bb-87cbe80e81f9 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:24.444536 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe52864-1317-4ca5-95bb-87cbe80e81f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:24.444903 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe52864-1317-4ca5-95bb-87cbe80e81f9 admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:24.445651 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe52864-1317-4ca5-95bb-87cbe80e81f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:24.449138 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe52864-1317-4ca5-95bb-87cbe80e81f9 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:24.565617 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 929/1853] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jun 29 11:25:24 2022] POST /identity/v3/users => generated 293 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:25:24.568382 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe52864-1317-4ca5-95bb-87cbe80e81f9 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:24.570202 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-4fe52864-1317-4ca5-95bb-87cbe80e81f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:25:24.597237 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 925/1854] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jun 29 11:25:24 2022] POST /identity/v3/projects => generated 379 bytes in 321 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:25:24.598480 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd3d070-af9f-4e41-a631-172e85342881 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:24.660916 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd3d070-af9f-4e41-a631-172e85342881 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:24.663298 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe52864-1317-4ca5-95bb-87cbe80e81f9 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:24.665688 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65116a50-dd14-448a-8f23-e3c4d2d4716b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:24.666512 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65116a50-dd14-448a-8f23-e3c4d2d4716b admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:24.666801 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65116a50-dd14-448a-8f23-e3c4d2d4716b admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:24.667083 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65116a50-dd14-448a-8f23-e3c4d2d4716b admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:24.667813 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65116a50-dd14-448a-8f23-e3c4d2d4716b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:24.669446 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65116a50-dd14-448a-8f23-e3c4d2d4716b admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:24.679460 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 926/1855] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:25:24 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 11:25:24.680829 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65116a50-dd14-448a-8f23-e3c4d2d4716b admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:24.690682 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65116a50-dd14-448a-8f23-e3c4d2d4716b admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:24.695625 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf72167-02ad-4acc-a706-5e189abc05a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:24.696907 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf72167-02ad-4acc-a706-5e189abc05a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:24.697195 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf72167-02ad-4acc-a706-5e189abc05a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:24.697476 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf72167-02ad-4acc-a706-5e189abc05a8 admin admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:24.698164 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf72167-02ad-4acc-a706-5e189abc05a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:24.700124 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf72167-02ad-4acc-a706-5e189abc05a8 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:24.721986 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9ad7ee-ca1b-4e75-8861-17aeb059e278 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f4bb9f989e4024a5da28dced646b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:24.724023 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9ad7ee-ca1b-4e75-8861-17aeb059e278 service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:24.724023 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9ad7ee-ca1b-4e75-8861-17aeb059e278 service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:24.724023 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9ad7ee-ca1b-4e75-8861-17aeb059e278 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:24.732270 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9ad7ee-ca1b-4e75-8861-17aeb059e278 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:24.733689 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9ad7ee-ca1b-4e75-8861-17aeb059e278 service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:24.743974 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 930/1856] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:25:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:25:24.744614 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9ad7ee-ca1b-4e75-8861-17aeb059e278 service neutron] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:24.761551 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9ad7ee-ca1b-4e75-8861-17aeb059e278 service neutron] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:24.769707 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c073c890-df30-48ef-8bf8-59b322076cf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:24.770406 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c073c890-df30-48ef-8bf8-59b322076cf1 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:24.770680 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 927/1857] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jun 29 11:25:24 2022] POST /identity/v3/users => generated 290 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:25:24.770784 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c073c890-df30-48ef-8bf8-59b322076cf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:24.774180 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c073c890-df30-48ef-8bf8-59b322076cf1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:24.774180 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c073c890-df30-48ef-8bf8-59b322076cf1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:24.774180 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c073c890-df30-48ef-8bf8-59b322076cf1 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:24.777292 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf72167-02ad-4acc-a706-5e189abc05a8 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:24.786115 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 931/1858] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:25:24 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 11:25:24.790635 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c073c890-df30-48ef-8bf8-59b322076cf1 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:24.794304 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf72167-02ad-4acc-a706-5e189abc05a8 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:24.803867 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbaf6e76-f4df-46ce-ab53-737d7adff9de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:24.804618 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaf6e76-f4df-46ce-ab53-737d7adff9de admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:24.804726 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c073c890-df30-48ef-8bf8-59b322076cf1 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:24.804931 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaf6e76-f4df-46ce-ab53-737d7adff9de admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:24.805968 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaf6e76-f4df-46ce-ab53-737d7adff9de admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:24.806480 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbaf6e76-f4df-46ce-ab53-737d7adff9de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:24.808251 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbaf6e76-f4df-46ce-ab53-737d7adff9de admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:24.810053 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f0cc95-02a9-4b64-aa1b-00dbf0f90276 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:24.810946 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f0cc95-02a9-4b64-aa1b-00dbf0f90276 admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:24.811236 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f0cc95-02a9-4b64-aa1b-00dbf0f90276 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:24.811520 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f0cc95-02a9-4b64-aa1b-00dbf0f90276 admin admin] PATH_INFO: `/v3/projects/717a5f10aba945cb889b19f0fb0cbb8d/users/5a23e00456fd430f8656b0ca4e6e8aac/roles/e1c88eebe911432cab6a07fbeb84653b` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:24.819738 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 928/1859] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:25:24 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 11:25:24.824298 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbaf6e76-f4df-46ce-ab53-737d7adff9de admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:24.831257 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f0cc95-02a9-4b64-aa1b-00dbf0f90276 admin admin] RBAC: Authorizing `identity:create_grant(project_id=717a5f10aba945cb889b19f0fb0cbb8d, user_id=5a23e00456fd430f8656b0ca4e6e8aac, role_id=e1c88eebe911432cab6a07fbeb84653b)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:24.834132 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbaf6e76-f4df-46ce-ab53-737d7adff9de admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:24.834275 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f0cc95-02a9-4b64-aa1b-00dbf0f90276 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:24.838903 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cc457c-e11c-43ad-a846-e363dc51ec0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:24.839609 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cc457c-e11c-43ad-a846-e363dc51ec0b admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:24.839934 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cc457c-e11c-43ad-a846-e363dc51ec0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:24.840216 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cc457c-e11c-43ad-a846-e363dc51ec0b admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:24.840966 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cc457c-e11c-43ad-a846-e363dc51ec0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:24.842643 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cc457c-e11c-43ad-a846-e363dc51ec0b admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:24.853352 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 929/1860] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:25:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 11:25:24.857555 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cc457c-e11c-43ad-a846-e363dc51ec0b admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:24.867271 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cc457c-e11c-43ad-a846-e363dc51ec0b admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:24.872636 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a0bb61-5a74-4af6-bfc1-cee839595db0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:24.873500 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a0bb61-5a74-4af6-bfc1-cee839595db0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:24.873785 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a0bb61-5a74-4af6-bfc1-cee839595db0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:24.874062 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a0bb61-5a74-4af6-bfc1-cee839595db0 admin admin] PATH_INFO: `/v3/projects/79c47ea37e4b4d3492b075b5a061a7e8/users/9aa4f531c61548b8ad0a1761e78d3434/roles/e1c88eebe911432cab6a07fbeb84653b` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:24.890322 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a0bb61-5a74-4af6-bfc1-cee839595db0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=79c47ea37e4b4d3492b075b5a061a7e8, user_id=9aa4f531c61548b8ad0a1761e78d3434, role_id=e1c88eebe911432cab6a07fbeb84653b)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:24.893040 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a0bb61-5a74-4af6-bfc1-cee839595db0 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:24.918153 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 932/1861] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jun 29 11:25:24 2022] PUT /identity/v3/projects/717a5f10aba945cb889b19f0fb0cbb8d/users/5a23e00456fd430f8656b0ca4e6e8aac/roles/e1c88eebe911432cab6a07fbeb84653b => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:25:24.927872 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a9808d-2de6-4ea3-bfe1-8531a2912005 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:24.928168 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a9808d-2de6-4ea3-bfe1-8531a2912005 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:24.928447 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a9808d-2de6-4ea3-bfe1-8531a2912005 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:24.954593 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-f8a9808d-2de6-4ea3-bfe1-8531a2912005 None None] MFA Rules not processed for user `5a23e00456fd430f8656b0ca4e6e8aac`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:25:24.955803 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-f8a9808d-2de6-4ea3-bfe1-8531a2912005 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:25:25.014671 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 930/1862] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jun 29 11:25:24 2022] PUT /identity/v3/projects/79c47ea37e4b4d3492b075b5a061a7e8/users/9aa4f531c61548b8ad0a1761e78d3434/roles/e1c88eebe911432cab6a07fbeb84653b => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 11:25:25.017241 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076c5fe3-85fb-4efc-a471-cf3486d46491 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:25.017709 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076c5fe3-85fb-4efc-a471-cf3486d46491 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:25.018174 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076c5fe3-85fb-4efc-a471-cf3486d46491 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:25.042532 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 933/1863] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:25:24 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 11:25:25.047710 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a9808d-2de6-4ea3-bfe1-8531a2912005 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:25.053871 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-076c5fe3-85fb-4efc-a471-cf3486d46491 None None] MFA Rules not processed for user `9aa4f531c61548b8ad0a1761e78d3434`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:25:25.055235 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-076c5fe3-85fb-4efc-a471-cf3486d46491 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:25:25.121469 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a9808d-2de6-4ea3-bfe1-8531a2912005 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:25.131511 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 931/1864] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:25:25 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:25:25.132596 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076c5fe3-85fb-4efc-a471-cf3486d46491 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:25.154524 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:25:25.190567 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1670a0d8-faf4-4285-b62d-bdfe64ad1d9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:25.191362 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1670a0d8-faf4-4285-b62d-bdfe64ad1d9c admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:25.191537 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1670a0d8-faf4-4285-b62d-bdfe64ad1d9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:25.191818 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1670a0d8-faf4-4285-b62d-bdfe64ad1d9c admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:25.192676 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1670a0d8-faf4-4285-b62d-bdfe64ad1d9c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:25.194477 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1670a0d8-faf4-4285-b62d-bdfe64ad1d9c admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:25.209910 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076c5fe3-85fb-4efc-a471-cf3486d46491 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:25.214526 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82516d2-5e43-4be9-a5db-320efbe15327 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:25.215336 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82516d2-5e43-4be9-a5db-320efbe15327 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:25.215629 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82516d2-5e43-4be9-a5db-320efbe15327 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:25.215954 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82516d2-5e43-4be9-a5db-320efbe15327 admin admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:25.216647 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82516d2-5e43-4be9-a5db-320efbe15327 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:25.218557 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82516d2-5e43-4be9-a5db-320efbe15327 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:25.286962 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 934/1865] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jun 29 11:25:25 2022] POST /identity/v3/projects => generated 383 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:25:25.286962 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1670a0d8-faf4-4285-b62d-bdfe64ad1d9c admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:25.287820 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-1670a0d8-faf4-4285-b62d-bdfe64ad1d9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:25:25.316899 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 932/1866] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jun 29 11:25:25 2022] POST /identity/v3/projects => generated 379 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 11:25:25.316899 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82516d2-5e43-4be9-a5db-320efbe15327 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:25.351293 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1670a0d8-faf4-4285-b62d-bdfe64ad1d9c admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:25.379340 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82516d2-5e43-4be9-a5db-320efbe15327 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:25.384119 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7a629c-a372-4a6f-beeb-aa5a052bbea4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:25.384792 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7a629c-a372-4a6f-beeb-aa5a052bbea4 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:25.385076 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7a629c-a372-4a6f-beeb-aa5a052bbea4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:25.385356 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7a629c-a372-4a6f-beeb-aa5a052bbea4 admin admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:25.386045 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7a629c-a372-4a6f-beeb-aa5a052bbea4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:25.388002 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7a629c-a372-4a6f-beeb-aa5a052bbea4 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:25.406166 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7418c5-7ba7-46f6-ab73-c70b2b12c37c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:25.407032 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7418c5-7ba7-46f6-ab73-c70b2b12c37c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:25.410604 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7418c5-7ba7-46f6-ab73-c70b2b12c37c admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:25.410604 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7418c5-7ba7-46f6-ab73-c70b2b12c37c admin admin] PATH_INFO: `/v3/projects/05d01361b1924cd4a77fb6c8bd887c58` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:25.410604 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7418c5-7ba7-46f6-ab73-c70b2b12c37c admin admin] RBAC: Authorizing `identity:delete_project(project_id=05d01361b1924cd4a77fb6c8bd887c58)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:25.410927 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7418c5-7ba7-46f6-ab73-c70b2b12c37c admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:25.464100 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 933/1867] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jun 29 11:25:25 2022] POST /identity/v3/users => generated 298 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:25:25.465367 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7a629c-a372-4a6f-beeb-aa5a052bbea4 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:25.476502 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7a629c-a372-4a6f-beeb-aa5a052bbea4 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:25.482211 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bdca57-dd3a-45b7-8a91-795ab412c99b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:25.483030 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bdca57-dd3a-45b7-8a91-795ab412c99b admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:25.483316 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bdca57-dd3a-45b7-8a91-795ab412c99b admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:25.483594 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bdca57-dd3a-45b7-8a91-795ab412c99b admin admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:25.484343 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bdca57-dd3a-45b7-8a91-795ab412c99b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:25.486275 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bdca57-dd3a-45b7-8a91-795ab412c99b admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:25.558584 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-0b7418c5-7ba7-46f6-ab73-c70b2b12c37c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '05d01361b1924cd4a77fb6c8bd887c58', 'request_id': 'req-0b7418c5-7ba7-46f6-ab73-c70b2b12c37c'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:25.564848 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-0b7418c5-7ba7-46f6-ab73-c70b2b12c37c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '05d01361b1924cd4a77fb6c8bd887c58', 'request_id': 'req-0b7418c5-7ba7-46f6-ab73-c70b2b12c37c'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:25.573000 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:25:25.577259 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-0b7418c5-7ba7-46f6-ab73-c70b2b12c37c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '05d01361b1924cd4a77fb6c8bd887c58', 'request_id': 'req-0b7418c5-7ba7-46f6-ab73-c70b2b12c37c'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:25.582483 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 934/1868] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jun 29 11:25:25 2022] POST /identity/v3/users => generated 290 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:25:25.582483 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bdca57-dd3a-45b7-8a91-795ab412c99b admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:25.590535 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bdca57-dd3a-45b7-8a91-795ab412c99b admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:25.648073 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-0b7418c5-7ba7-46f6-ab73-c70b2b12c37c admin admin] The token cache is being invalidate because project 05d01361b1924cd4a77fb6c8bd887c58 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 11:25:25.648364 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-0b7418c5-7ba7-46f6-ab73-c70b2b12c37c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:25.650027 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-0b7418c5-7ba7-46f6-ab73-c70b2b12c37c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:25.650953 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77f80a0-ae5f-4adc-a82c-2d526084c0ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:25.651629 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77f80a0-ae5f-4adc-a82c-2d526084c0ce admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:25.651946 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77f80a0-ae5f-4adc-a82c-2d526084c0ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:25.652225 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77f80a0-ae5f-4adc-a82c-2d526084c0ce admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:25.652986 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77f80a0-ae5f-4adc-a82c-2d526084c0ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:25.653360 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 935/1869] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jun 29 11:25:25 2022] DELETE /identity/v3/projects/05d01361b1924cd4a77fb6c8bd887c58 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:25:25.654548 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7418c5-7ba7-46f6-ab73-c70b2b12c37c admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:25.654687 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77f80a0-ae5f-4adc-a82c-2d526084c0ce admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:25.657017 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-0b7418c5-7ba7-46f6-ab73-c70b2b12c37c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:25:25.668097 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 935/1870] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:25:25 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 11:25:25.668097 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77f80a0-ae5f-4adc-a82c-2d526084c0ce admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:25.674669 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-b77f80a0-ae5f-4adc-a82c-2d526084c0ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:25:25.674820 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7418c5-7ba7-46f6-ab73-c70b2b12c37c admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:25.682386 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67960fd2-fd76-4bda-87bb-afb3bf2ce6ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:25.682785 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77f80a0-ae5f-4adc-a82c-2d526084c0ce admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:25.683064 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67960fd2-fd76-4bda-87bb-afb3bf2ce6ec admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:25.683348 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67960fd2-fd76-4bda-87bb-afb3bf2ce6ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:25.683714 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67960fd2-fd76-4bda-87bb-afb3bf2ce6ec admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:25.684928 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67960fd2-fd76-4bda-87bb-afb3bf2ce6ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:25.687484 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67960fd2-fd76-4bda-87bb-afb3bf2ce6ec admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:25.687704 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb98b9e-30bd-45bf-ab03-652082a465ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:25.688619 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb98b9e-30bd-45bf-ab03-652082a465ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:25.688913 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb98b9e-30bd-45bf-ab03-652082a465ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:25.689393 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb98b9e-30bd-45bf-ab03-652082a465ad admin admin] PATH_INFO: `/v3/users/fcf645ee87d5473db4a4a57533dc7c52` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:25.690630 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb98b9e-30bd-45bf-ab03-652082a465ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=fcf645ee87d5473db4a4a57533dc7c52)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:25.692692 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb98b9e-30bd-45bf-ab03-652082a465ad admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:25.704011 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 936/1871] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:25:25 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 11:25:25.704011 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67960fd2-fd76-4bda-87bb-afb3bf2ce6ec admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:25.705645 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-67960fd2-fd76-4bda-87bb-afb3bf2ce6ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:25:25.720111 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67960fd2-fd76-4bda-87bb-afb3bf2ce6ec admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:25.724756 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f766ad-7bb7-4b0c-abe5-880e801d0c90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:25.725642 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f766ad-7bb7-4b0c-abe5-880e801d0c90 admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:25.725923 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f766ad-7bb7-4b0c-abe5-880e801d0c90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:25.726201 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f766ad-7bb7-4b0c-abe5-880e801d0c90 admin admin] PATH_INFO: `/v3/projects/19943531ab424e9a93d8813017e9db58/users/ab8685ef23404ded9e4480f2b3ea3320/roles/e95c2ce63e114cc19a9dda6ad21e794f` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:25.741686 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f766ad-7bb7-4b0c-abe5-880e801d0c90 admin admin] RBAC: Authorizing `identity:create_grant(project_id=19943531ab424e9a93d8813017e9db58, user_id=ab8685ef23404ded9e4480f2b3ea3320, role_id=e95c2ce63e114cc19a9dda6ad21e794f)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:25.744442 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f766ad-7bb7-4b0c-abe5-880e801d0c90 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:25.809380 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 937/1872] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jun 29 11:25:25 2022] PUT /identity/v3/projects/19943531ab424e9a93d8813017e9db58/users/ab8685ef23404ded9e4480f2b3ea3320/roles/e95c2ce63e114cc19a9dda6ad21e794f => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:25:25.810359 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f766ad-7bb7-4b0c-abe5-880e801d0c90 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:25.868211 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-2cb98b9e-30bd-45bf-ab03-652082a465ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fcf645ee87d5473db4a4a57533dc7c52'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:25.872326 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f766ad-7bb7-4b0c-abe5-880e801d0c90 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:25.877239 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687a928a-677e-410f-a982-a8fb47bf8c2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:25.878000 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687a928a-677e-410f-a982-a8fb47bf8c2b admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:25.878256 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687a928a-677e-410f-a982-a8fb47bf8c2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:25.878535 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687a928a-677e-410f-a982-a8fb47bf8c2b admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:25.879285 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687a928a-677e-410f-a982-a8fb47bf8c2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:25.880912 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687a928a-677e-410f-a982-a8fb47bf8c2b admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:25.891503 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 938/1873] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:25:25 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 11:25:25.892337 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687a928a-677e-410f-a982-a8fb47bf8c2b admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:25.902630 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687a928a-677e-410f-a982-a8fb47bf8c2b admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:25.907294 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8220fcc4-1163-49cb-b2a9-41db2f0df802 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:25.907988 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8220fcc4-1163-49cb-b2a9-41db2f0df802 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:25.908271 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8220fcc4-1163-49cb-b2a9-41db2f0df802 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:25.908546 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8220fcc4-1163-49cb-b2a9-41db2f0df802 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:25.909268 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8220fcc4-1163-49cb-b2a9-41db2f0df802 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:25.910861 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8220fcc4-1163-49cb-b2a9-41db2f0df802 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:25.929836 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 939/1874] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:25:25 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 11:25:25.929836 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8220fcc4-1163-49cb-b2a9-41db2f0df802 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:25.930697 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-2cb98b9e-30bd-45bf-ab03-652082a465ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fcf645ee87d5473db4a4a57533dc7c52'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:25.930697 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-2cb98b9e-30bd-45bf-ab03-652082a465ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fcf645ee87d5473db4a4a57533dc7c52'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:25.943591 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8220fcc4-1163-49cb-b2a9-41db2f0df802 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:25.949676 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-2cb98b9e-30bd-45bf-ab03-652082a465ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fcf645ee87d5473db4a4a57533dc7c52'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:25.950698 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-2cb98b9e-30bd-45bf-ab03-652082a465ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fcf645ee87d5473db4a4a57533dc7c52'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:25.951372 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd2eb63-e75d-43ae-9b40-26b5b00ac967 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:25.952923 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd2eb63-e75d-43ae-9b40-26b5b00ac967 admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:25.953390 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd2eb63-e75d-43ae-9b40-26b5b00ac967 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:25.953836 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd2eb63-e75d-43ae-9b40-26b5b00ac967 admin admin] PATH_INFO: `/v3/projects/e932b5a8272d4a579a33df0c7d2012ba/users/a46cd446b7da474c8602395140bbf413/roles/e1c88eebe911432cab6a07fbeb84653b` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:25.966132 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 936/1875] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jun 29 11:25:25 2022] DELETE /identity/v3/users/fcf645ee87d5473db4a4a57533dc7c52 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:25:25.970870 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb98b9e-30bd-45bf-ab03-652082a465ad admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:25.970870 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-2cb98b9e-30bd-45bf-ab03-652082a465ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:25:25.974577 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd2eb63-e75d-43ae-9b40-26b5b00ac967 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e932b5a8272d4a579a33df0c7d2012ba, user_id=a46cd446b7da474c8602395140bbf413, role_id=e1c88eebe911432cab6a07fbeb84653b)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:25.978917 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd2eb63-e75d-43ae-9b40-26b5b00ac967 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:26.034846 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb98b9e-30bd-45bf-ab03-652082a465ad admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:26.039594 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618d4695-adbf-4f64-99be-da328903858d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:26.040524 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618d4695-adbf-4f64-99be-da328903858d admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:26.040795 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618d4695-adbf-4f64-99be-da328903858d admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:26.041068 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618d4695-adbf-4f64-99be-da328903858d admin admin] PATH_INFO: `/v3/projects/19943531ab424e9a93d8813017e9db58/users/ab8685ef23404ded9e4480f2b3ea3320/roles/e95c2ce63e114cc19a9dda6ad21e794f` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:26.043062 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618d4695-adbf-4f64-99be-da328903858d admin admin] RBAC: Authorizing `identity:create_grant(project_id=19943531ab424e9a93d8813017e9db58, user_id=ab8685ef23404ded9e4480f2b3ea3320, role_id=e95c2ce63e114cc19a9dda6ad21e794f)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:26.045781 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618d4695-adbf-4f64-99be-da328903858d admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:26.056006 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 937/1876] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jun 29 11:25:25 2022] PUT /identity/v3/projects/19943531ab424e9a93d8813017e9db58/users/ab8685ef23404ded9e4480f2b3ea3320/roles/e95c2ce63e114cc19a9dda6ad21e794f => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:25:26.067192 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3836f30e-abc7-4cb0-bcf6-bb59f69989b4 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:26.071315 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3836f30e-abc7-4cb0-bcf6-bb59f69989b4 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:26.071315 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3836f30e-abc7-4cb0-bcf6-bb59f69989b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:26.095535 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 940/1877] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jun 29 11:25:25 2022] PUT /identity/v3/projects/e932b5a8272d4a579a33df0c7d2012ba/users/a46cd446b7da474c8602395140bbf413/roles/e1c88eebe911432cab6a07fbeb84653b => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:25:26.096771 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-3836f30e-abc7-4cb0-bcf6-bb59f69989b4 None None] MFA Rules not processed for user `ab8685ef23404ded9e4480f2b3ea3320`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:25:26.097997 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-3836f30e-abc7-4cb0-bcf6-bb59f69989b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:25:26.105716 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2758f3d9-b3dc-4826-b3b2-671d4840e054 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:26.106006 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2758f3d9-b3dc-4826-b3b2-671d4840e054 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:26.106286 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2758f3d9-b3dc-4826-b3b2-671d4840e054 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:26.118054 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:25:26.143564 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-2758f3d9-b3dc-4826-b3b2-671d4840e054 None None] MFA Rules not processed for user `a46cd446b7da474c8602395140bbf413`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:25:26.144771 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-2758f3d9-b3dc-4826-b3b2-671d4840e054 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:25:26.170967 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:INSERT=5,keystone:DELETE=1,keystone:SELECT=94 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:25:26.172140 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 938/1878] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:25:26 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 11:25:26.185754 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3836f30e-abc7-4cb0-bcf6-bb59f69989b4 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:26.187990 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-3836f30e-abc7-4cb0-bcf6-bb59f69989b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:25:26.239751 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 941/1879] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:25:26 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:25:26.246972 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2758f3d9-b3dc-4826-b3b2-671d4840e054 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:26.248881 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-2758f3d9-b3dc-4826-b3b2-671d4840e054 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:25:26.270977 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3836f30e-abc7-4cb0-bcf6-bb59f69989b4 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:26.274440 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9065dc62-d53c-480a-b590-326df5e3b786 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff7931b75a924394b1af733febce0834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:26.275276 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9065dc62-d53c-480a-b590-326df5e3b786 service cinder] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:26.275525 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9065dc62-d53c-480a-b590-326df5e3b786 service cinder] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:26.275900 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9065dc62-d53c-480a-b590-326df5e3b786 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:26.278240 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-9065dc62-d53c-480a-b590-326df5e3b786 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:25:26.334539 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2758f3d9-b3dc-4826-b3b2-671d4840e054 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:26.340331 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3bd3f6-8684-43ea-bf91-0b8ccd7f4bce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:26.341002 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3bd3f6-8684-43ea-bf91-0b8ccd7f4bce admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:26.341290 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3bd3f6-8684-43ea-bf91-0b8ccd7f4bce admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:26.341568 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3bd3f6-8684-43ea-bf91-0b8ccd7f4bce admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:26.342271 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3bd3f6-8684-43ea-bf91-0b8ccd7f4bce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:26.344723 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3bd3f6-8684-43ea-bf91-0b8ccd7f4bce admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:26.345463 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9065dc62-d53c-480a-b590-326df5e3b786 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:26.346913 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9065dc62-d53c-480a-b590-326df5e3b786 service cinder] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:26.356431 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 939/1880] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:25:26 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:25:26.454284 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 942/1881] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jun 29 11:25:26 2022] POST /identity/v3/projects => generated 379 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:25:26.458508 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9065dc62-d53c-480a-b590-326df5e3b786 service cinder] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:26.469365 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:25:26.475128 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9065dc62-d53c-480a-b590-326df5e3b786 service cinder] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:26.476385 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3bd3f6-8684-43ea-bf91-0b8ccd7f4bce admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:26.479861 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-6a3bd3f6-8684-43ea-bf91-0b8ccd7f4bce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:25:26.543948 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f98fd2-ab7b-4035-811c-30c54f64fe3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:26.544622 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f98fd2-ab7b-4035-811c-30c54f64fe3c admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:26.544914 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f98fd2-ab7b-4035-811c-30c54f64fe3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:26.545201 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f98fd2-ab7b-4035-811c-30c54f64fe3c admin admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:26.545900 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f98fd2-ab7b-4035-811c-30c54f64fe3c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:26.549053 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f98fd2-ab7b-4035-811c-30c54f64fe3c admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:26.563681 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3bd3f6-8684-43ea-bf91-0b8ccd7f4bce admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:26.566055 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:25:26.570534 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6828e903-20dc-4d31-9a07-12e3792f5a0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:26.571630 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6828e903-20dc-4d31-9a07-12e3792f5a0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:26.572003 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6828e903-20dc-4d31-9a07-12e3792f5a0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:26.572460 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6828e903-20dc-4d31-9a07-12e3792f5a0b admin admin] PATH_INFO: `/v3/projects/d80466559f8249c0996641dce576e034` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:26.573763 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6828e903-20dc-4d31-9a07-12e3792f5a0b admin admin] RBAC: Authorizing `identity:delete_project(project_id=d80466559f8249c0996641dce576e034)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:26.575730 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6828e903-20dc-4d31-9a07-12e3792f5a0b admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:26.666050 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 940/1882] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jun 29 11:25:26 2022] POST /identity/v3/users => generated 296 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:25:26.667039 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f98fd2-ab7b-4035-811c-30c54f64fe3c admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:26.676746 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f98fd2-ab7b-4035-811c-30c54f64fe3c admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:26.682132 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6654a1-3519-4490-907e-9325ddd44059 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:26.682781 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6654a1-3519-4490-907e-9325ddd44059 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:26.683070 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6654a1-3519-4490-907e-9325ddd44059 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:26.683352 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6654a1-3519-4490-907e-9325ddd44059 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:26.684122 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6654a1-3519-4490-907e-9325ddd44059 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:26.685863 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6654a1-3519-4490-907e-9325ddd44059 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:26.711874 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 941/1883] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:25:26 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 11:25:26.713874 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6654a1-3519-4490-907e-9325ddd44059 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:26.719176 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-6828e903-20dc-4d31-9a07-12e3792f5a0b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd80466559f8249c0996641dce576e034', 'request_id': 'req-6828e903-20dc-4d31-9a07-12e3792f5a0b'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:26.725049 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6654a1-3519-4490-907e-9325ddd44059 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:26.726049 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-6828e903-20dc-4d31-9a07-12e3792f5a0b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd80466559f8249c0996641dce576e034', 'request_id': 'req-6828e903-20dc-4d31-9a07-12e3792f5a0b'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:26.733820 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-6828e903-20dc-4d31-9a07-12e3792f5a0b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd80466559f8249c0996641dce576e034', 'request_id': 'req-6828e903-20dc-4d31-9a07-12e3792f5a0b'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:26.791257 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d28f14-6bff-4508-8cdf-96996156c37f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:26.792413 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d28f14-6bff-4508-8cdf-96996156c37f admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:26.792677 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d28f14-6bff-4508-8cdf-96996156c37f admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:26.792956 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d28f14-6bff-4508-8cdf-96996156c37f admin admin] PATH_INFO: `/v3/projects/a6dc7134a91f41ac90807a80195bbaf7/users/0347a1774de84f9e943d0e3297b2f46a/roles/e95c2ce63e114cc19a9dda6ad21e794f` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:26.796783 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=86,keystone:INSERT=7,keystone:DELETE=7 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:25:26.798055 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-6828e903-20dc-4d31-9a07-12e3792f5a0b admin admin] The token cache is being invalidate because project d80466559f8249c0996641dce576e034 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 11:25:26.798445 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-6828e903-20dc-4d31-9a07-12e3792f5a0b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:26.799286 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-6828e903-20dc-4d31-9a07-12e3792f5a0b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:26.801157 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 943/1884] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jun 29 11:25:26 2022] DELETE /identity/v3/projects/d80466559f8249c0996641dce576e034 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:25:26.807873 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2e3354-52ee-4ae6-8185-e246d2347626 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:26.808245 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2e3354-52ee-4ae6-8185-e246d2347626 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:26.809606 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2e3354-52ee-4ae6-8185-e246d2347626 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:26.815053 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d28f14-6bff-4508-8cdf-96996156c37f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6dc7134a91f41ac90807a80195bbaf7, user_id=0347a1774de84f9e943d0e3297b2f46a, role_id=e95c2ce63e114cc19a9dda6ad21e794f)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:26.818116 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d28f14-6bff-4508-8cdf-96996156c37f admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:26.831764 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-ad2e3354-52ee-4ae6-8185-e246d2347626 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:25:26.833188 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-ad2e3354-52ee-4ae6-8185-e246d2347626 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:25:26.842187 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 944/1885] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:25:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:25:26.850068 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea5e300-a2f4-4d23-8741-cf80feec1571 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:26.850363 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea5e300-a2f4-4d23-8741-cf80feec1571 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:26.850648 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea5e300-a2f4-4d23-8741-cf80feec1571 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:26.867252 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:25:26.869973 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-bea5e300-a2f4-4d23-8741-cf80feec1571 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:25:26.871317 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-bea5e300-a2f4-4d23-8741-cf80feec1571 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:25:26.888148 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 945/1886] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:25:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 11:25:26.898819 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea5e300-a2f4-4d23-8741-cf80feec1571 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:26.911737 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 942/1887] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jun 29 11:25:26 2022] PUT /identity/v3/projects/a6dc7134a91f41ac90807a80195bbaf7/users/0347a1774de84f9e943d0e3297b2f46a/roles/e95c2ce63e114cc19a9dda6ad21e794f => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 11:25:26.913157 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d28f14-6bff-4508-8cdf-96996156c37f admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:26.916269 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea5e300-a2f4-4d23-8741-cf80feec1571 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:26.933799 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-49d28f14-6bff-4508-8cdf-96996156c37f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:25:26.984910 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7af685-d89b-4e38-b26c-c0968fb7a463 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:26.985680 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7af685-d89b-4e38-b26c-c0968fb7a463 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:26.986039 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7af685-d89b-4e38-b26c-c0968fb7a463 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:26.986390 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7af685-d89b-4e38-b26c-c0968fb7a463 admin admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:26.987220 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7af685-d89b-4e38-b26c-c0968fb7a463 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:25:26.987630 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7af685-d89b-4e38-b26c-c0968fb7a463 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:26.989422 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7af685-d89b-4e38-b26c-c0968fb7a463 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:27.001998 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 946/1888] 10.222.0.44 () {58 vars in 1152 bytes} [Wed Jun 29 11:25:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:25:27.006318 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7af685-d89b-4e38-b26c-c0968fb7a463 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:27.011364 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d28f14-6bff-4508-8cdf-96996156c37f admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:27.018319 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7af685-d89b-4e38-b26c-c0968fb7a463 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:27.019520 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc27607-9349-46df-9730-e24551102f92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:27.020320 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc27607-9349-46df-9730-e24551102f92 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:27.020545 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc27607-9349-46df-9730-e24551102f92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:27.020769 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc27607-9349-46df-9730-e24551102f92 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:27.021498 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc27607-9349-46df-9730-e24551102f92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:27.023107 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc27607-9349-46df-9730-e24551102f92 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:27.025785 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784bb8d9-9731-4170-ad69-266a8d54e282 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:27.025785 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784bb8d9-9731-4170-ad69-266a8d54e282 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:27.026091 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784bb8d9-9731-4170-ad69-266a8d54e282 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:27.026369 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784bb8d9-9731-4170-ad69-266a8d54e282 admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:27.027059 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784bb8d9-9731-4170-ad69-266a8d54e282 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:27.029131 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784bb8d9-9731-4170-ad69-266a8d54e282 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:27.037143 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 943/1889] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:25:26 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 11:25:27.041578 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc27607-9349-46df-9730-e24551102f92 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:27.052457 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc27607-9349-46df-9730-e24551102f92 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:27.057427 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cd7eec-391e-4e86-ab38-e9ae576b7a26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:27.058371 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cd7eec-391e-4e86-ab38-e9ae576b7a26 admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:27.067920 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cd7eec-391e-4e86-ab38-e9ae576b7a26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:27.067920 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cd7eec-391e-4e86-ab38-e9ae576b7a26 admin admin] PATH_INFO: `/v3/projects/a6dc7134a91f41ac90807a80195bbaf7/users/0347a1774de84f9e943d0e3297b2f46a/roles/e95c2ce63e114cc19a9dda6ad21e794f` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:27.067920 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cd7eec-391e-4e86-ab38-e9ae576b7a26 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6dc7134a91f41ac90807a80195bbaf7, user_id=0347a1774de84f9e943d0e3297b2f46a, role_id=e95c2ce63e114cc19a9dda6ad21e794f)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:27.067920 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cd7eec-391e-4e86-ab38-e9ae576b7a26 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:27.075239 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 944/1890] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jun 29 11:25:27 2022] PUT /identity/v3/projects/a6dc7134a91f41ac90807a80195bbaf7/users/0347a1774de84f9e943d0e3297b2f46a/roles/e95c2ce63e114cc19a9dda6ad21e794f => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 11:25:27.085100 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e2fab8-44f0-40df-9141-1fd143ed259a None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:27.085358 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e2fab8-44f0-40df-9141-1fd143ed259a None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:27.085635 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e2fab8-44f0-40df-9141-1fd143ed259a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:27.113640 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-98e2fab8-44f0-40df-9141-1fd143ed259a None None] MFA Rules not processed for user `0347a1774de84f9e943d0e3297b2f46a`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:25:27.114874 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-98e2fab8-44f0-40df-9141-1fd143ed259a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:25:27.123205 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 947/1891] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jun 29 11:25:27 2022] POST /identity/v3/projects => generated 377 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:25:27.124258 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784bb8d9-9731-4170-ad69-266a8d54e282 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:27.198700 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784bb8d9-9731-4170-ad69-266a8d54e282 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:27.203912 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589f0779-4690-48e4-865c-45f4bb2439c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:27.204582 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589f0779-4690-48e4-865c-45f4bb2439c6 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:27.204865 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589f0779-4690-48e4-865c-45f4bb2439c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:27.205138 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589f0779-4690-48e4-865c-45f4bb2439c6 admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:27.205655 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 945/1892] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:25:27 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 11:25:27.205880 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589f0779-4690-48e4-865c-45f4bb2439c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:27.207726 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589f0779-4690-48e4-865c-45f4bb2439c6 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:27.217325 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e2fab8-44f0-40df-9141-1fd143ed259a None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:27.218791 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-98e2fab8-44f0-40df-9141-1fd143ed259a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:25:27.257073 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:25:27.277751 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e2fab8-44f0-40df-9141-1fd143ed259a None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:27.305570 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 948/1893] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jun 29 11:25:27 2022] POST /identity/v3/users => generated 289 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:25:27.306686 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589f0779-4690-48e4-865c-45f4bb2439c6 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:27.316392 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589f0779-4690-48e4-865c-45f4bb2439c6 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:27.321693 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf41e2e-0354-4c7a-b22e-924d1519cb9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:27.322371 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf41e2e-0354-4c7a-b22e-924d1519cb9d admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:27.322658 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf41e2e-0354-4c7a-b22e-924d1519cb9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:27.322940 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf41e2e-0354-4c7a-b22e-924d1519cb9d admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:27.323717 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf41e2e-0354-4c7a-b22e-924d1519cb9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:27.325500 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf41e2e-0354-4c7a-b22e-924d1519cb9d admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:27.327232 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9da44e1-8169-4b80-81e9-a6a8c392a3d7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff7931b75a924394b1af733febce0834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:27.327911 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9da44e1-8169-4b80-81e9-a6a8c392a3d7 service cinder] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:27.328199 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9da44e1-8169-4b80-81e9-a6a8c392a3d7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:27.328477 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9da44e1-8169-4b80-81e9-a6a8c392a3d7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:27.335771 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 949/1894] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:25:27 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 11:25:27.340157 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf41e2e-0354-4c7a-b22e-924d1519cb9d admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:27.349779 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf41e2e-0354-4c7a-b22e-924d1519cb9d admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:27.354515 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4bace1-c2e8-4548-8ef9-d382b904f281 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:27.355152 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4bace1-c2e8-4548-8ef9-d382b904f281 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:27.355431 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4bace1-c2e8-4548-8ef9-d382b904f281 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:27.355702 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4bace1-c2e8-4548-8ef9-d382b904f281 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:27.356439 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4bace1-c2e8-4548-8ef9-d382b904f281 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:27.358064 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4bace1-c2e8-4548-8ef9-d382b904f281 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:27.368597 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 950/1895] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:25:27 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 11:25:27.372639 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4bace1-c2e8-4548-8ef9-d382b904f281 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:27.378164 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:INSERT=2,keystone:DELETE=3,keystone:SELECT=95 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:25:27.382164 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4bace1-c2e8-4548-8ef9-d382b904f281 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:27.387174 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173589a9-d448-4f7c-8c2f-cd73bd0f53a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:27.388069 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173589a9-d448-4f7c-8c2f-cd73bd0f53a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:27.388350 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173589a9-d448-4f7c-8c2f-cd73bd0f53a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:27.388627 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173589a9-d448-4f7c-8c2f-cd73bd0f53a0 admin admin] PATH_INFO: `/v3/projects/4544fce0292645b481f7939f6f4ac05a/users/86ecc69eb95b4533b29d710b5c6c804c/roles/e1c88eebe911432cab6a07fbeb84653b` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:27.393860 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9da44e1-8169-4b80-81e9-a6a8c392a3d7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:27.395322 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9da44e1-8169-4b80-81e9-a6a8c392a3d7 service cinder] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:27.403532 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173589a9-d448-4f7c-8c2f-cd73bd0f53a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4544fce0292645b481f7939f6f4ac05a, user_id=86ecc69eb95b4533b29d710b5c6c804c, role_id=e1c88eebe911432cab6a07fbeb84653b)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:27.405659 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 946/1896] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:25:27 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 11:25:27.406251 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173589a9-d448-4f7c-8c2f-cd73bd0f53a0 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:27.496645 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 951/1897] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jun 29 11:25:27 2022] PUT /identity/v3/projects/4544fce0292645b481f7939f6f4ac05a/users/86ecc69eb95b4533b29d710b5c6c804c/roles/e1c88eebe911432cab6a07fbeb84653b => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:25:27.507855 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1001ecb-c916-4f96-8ad3-2666c97596c0 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:27.508684 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1001ecb-c916-4f96-8ad3-2666c97596c0 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:27.508977 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1001ecb-c916-4f96-8ad3-2666c97596c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:27.551353 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-a1001ecb-c916-4f96-8ad3-2666c97596c0 None None] MFA Rules not processed for user `86ecc69eb95b4533b29d710b5c6c804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:25:27.553168 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-a1001ecb-c916-4f96-8ad3-2666c97596c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:25:27.572813 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173589a9-d448-4f7c-8c2f-cd73bd0f53a0 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:27.575246 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-173589a9-d448-4f7c-8c2f-cd73bd0f53a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:25:27.636406 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 947/1898] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:25:27 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:25:27.640868 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1001ecb-c916-4f96-8ad3-2666c97596c0 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:27.652690 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173589a9-d448-4f7c-8c2f-cd73bd0f53a0 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:27.657777 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a282ac8-e1ba-43dc-859b-fa85ef21187a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff7931b75a924394b1af733febce0834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:27.659032 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a282ac8-e1ba-43dc-859b-fa85ef21187a service cinder] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:27.660247 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a282ac8-e1ba-43dc-859b-fa85ef21187a service cinder] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:27.660714 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a282ac8-e1ba-43dc-859b-fa85ef21187a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:27.662953 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-5a282ac8-e1ba-43dc-859b-fa85ef21187a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:25:27.750770 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1001ecb-c916-4f96-8ad3-2666c97596c0 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:27.751925 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a282ac8-e1ba-43dc-859b-fa85ef21187a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:27.753991 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a282ac8-e1ba-43dc-859b-fa85ef21187a service cinder] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:27.756404 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073924da-cf8f-4e1c-8881-25587f250f50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:27.757317 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073924da-cf8f-4e1c-8881-25587f250f50 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:27.757831 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073924da-cf8f-4e1c-8881-25587f250f50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:27.758987 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073924da-cf8f-4e1c-8881-25587f250f50 admin admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:27.760024 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073924da-cf8f-4e1c-8881-25587f250f50 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:27.762267 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073924da-cf8f-4e1c-8881-25587f250f50 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:27.766579 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 952/1899] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:25:27 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 11:25:27.846769 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 948/1900] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jun 29 11:25:27 2022] POST /identity/v3/projects => generated 377 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 11:25:27.847953 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a282ac8-e1ba-43dc-859b-fa85ef21187a service cinder] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:27.861222 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a282ac8-e1ba-43dc-859b-fa85ef21187a service cinder] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:27.952865 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72cac1b-5807-488c-adbb-c37db5eab933 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:27.953791 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72cac1b-5807-488c-adbb-c37db5eab933 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:27.958799 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72cac1b-5807-488c-adbb-c37db5eab933 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:27.959170 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72cac1b-5807-488c-adbb-c37db5eab933 admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:27.960018 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72cac1b-5807-488c-adbb-c37db5eab933 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:27.962264 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72cac1b-5807-488c-adbb-c37db5eab933 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:28.084642 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 953/1901] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jun 29 11:25:27 2022] POST /identity/v3/users => generated 295 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 11:25:28.088725 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073924da-cf8f-4e1c-8881-25587f250f50 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:28.101619 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073924da-cf8f-4e1c-8881-25587f250f50 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:28.108017 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d94017-f82b-4330-9563-9e4c74d0d5fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:28.108885 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d94017-f82b-4330-9563-9e4c74d0d5fa admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:28.109244 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d94017-f82b-4330-9563-9e4c74d0d5fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:28.109595 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d94017-f82b-4330-9563-9e4c74d0d5fa admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:28.110429 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d94017-f82b-4330-9563-9e4c74d0d5fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:28.112319 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d94017-f82b-4330-9563-9e4c74d0d5fa admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:28.140086 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 949/1902] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:25:28 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 11:25:28.141479 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72cac1b-5807-488c-adbb-c37db5eab933 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:28.153314 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72cac1b-5807-488c-adbb-c37db5eab933 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:28.158949 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e5894a-1709-4942-8c57-98541be641b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:28.160384 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e5894a-1709-4942-8c57-98541be641b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:28.160671 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e5894a-1709-4942-8c57-98541be641b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:28.160953 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e5894a-1709-4942-8c57-98541be641b3 admin admin] PATH_INFO: `/v3/projects/3061d9b3ca8a4a3988dfc8c4578c7cf0/users/883f1daa527b4acf8c4d15010ac405c0/roles/e95c2ce63e114cc19a9dda6ad21e794f` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:28.165647 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:25:28.182626 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e5894a-1709-4942-8c57-98541be641b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3061d9b3ca8a4a3988dfc8c4578c7cf0, user_id=883f1daa527b4acf8c4d15010ac405c0, role_id=e95c2ce63e114cc19a9dda6ad21e794f)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:28.185391 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e5894a-1709-4942-8c57-98541be641b3 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:28.211806 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:25:28.274717 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d94017-f82b-4330-9563-9e4c74d0d5fa admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:28.275154 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 954/1903] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jun 29 11:25:28 2022] PUT /identity/v3/projects/3061d9b3ca8a4a3988dfc8c4578c7cf0/users/883f1daa527b4acf8c4d15010ac405c0/roles/e95c2ce63e114cc19a9dda6ad21e794f => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:25:28.285742 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d94017-f82b-4330-9563-9e4c74d0d5fa admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:28.353799 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52876a73-e145-41f3-8b32-7023f144718e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:28.354561 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52876a73-e145-41f3-8b32-7023f144718e admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:28.354809 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52876a73-e145-41f3-8b32-7023f144718e admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:28.355001 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52876a73-e145-41f3-8b32-7023f144718e admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:28.356238 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52876a73-e145-41f3-8b32-7023f144718e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:28.357850 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52876a73-e145-41f3-8b32-7023f144718e admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:28.373351 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 950/1904] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:25:28 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 11:25:28.374817 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e5894a-1709-4942-8c57-98541be641b3 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:28.385795 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e5894a-1709-4942-8c57-98541be641b3 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:28.391062 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ef63a7-b2bc-4170-bd2a-9476a86120a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:28.392433 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ef63a7-b2bc-4170-bd2a-9476a86120a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:28.392433 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ef63a7-b2bc-4170-bd2a-9476a86120a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:28.392621 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ef63a7-b2bc-4170-bd2a-9476a86120a4 admin admin] PATH_INFO: `/v3/projects/3061d9b3ca8a4a3988dfc8c4578c7cf0/users/883f1daa527b4acf8c4d15010ac405c0/roles/e95c2ce63e114cc19a9dda6ad21e794f` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:28.394517 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ef63a7-b2bc-4170-bd2a-9476a86120a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3061d9b3ca8a4a3988dfc8c4578c7cf0, user_id=883f1daa527b4acf8c4d15010ac405c0, role_id=e95c2ce63e114cc19a9dda6ad21e794f)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:28.397263 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ef63a7-b2bc-4170-bd2a-9476a86120a4 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:28.407984 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 955/1905] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jun 29 11:25:28 2022] PUT /identity/v3/projects/3061d9b3ca8a4a3988dfc8c4578c7cf0/users/883f1daa527b4acf8c4d15010ac405c0/roles/e95c2ce63e114cc19a9dda6ad21e794f => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:25:28.418840 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3748070-a6c5-411e-a933-d1288b6f280a None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:28.419273 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3748070-a6c5-411e-a933-d1288b6f280a None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:28.419564 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3748070-a6c5-411e-a933-d1288b6f280a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:28.455321 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-a3748070-a6c5-411e-a933-d1288b6f280a None None] MFA Rules not processed for user `883f1daa527b4acf8c4d15010ac405c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:25:28.456579 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-a3748070-a6c5-411e-a933-d1288b6f280a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:25:28.528100 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 951/1906] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:25:28 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:25:28.539053 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ef63a7-b2bc-4170-bd2a-9476a86120a4 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:28.598717 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ef63a7-b2bc-4170-bd2a-9476a86120a4 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:28.602926 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33eefa3d-2624-4586-ac85-d0112f38f462 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff7931b75a924394b1af733febce0834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:28.603574 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33eefa3d-2624-4586-ac85-d0112f38f462 service cinder] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:28.603881 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33eefa3d-2624-4586-ac85-d0112f38f462 service cinder] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:28.604165 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33eefa3d-2624-4586-ac85-d0112f38f462 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:28.659445 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33eefa3d-2624-4586-ac85-d0112f38f462 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:28.660895 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33eefa3d-2624-4586-ac85-d0112f38f462 service cinder] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:28.670194 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 956/1907] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:25:28 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:25:28.929593 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:25:29.125345 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:25:29.479860 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:25:29.956172 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:25:37.647211 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3748070-a6c5-411e-a933-d1288b6f280a None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:37.658237 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3748070-a6c5-411e-a933-d1288b6f280a None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:37.662657 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff56618-d8f5-4622-a4f9-5a37d01cab9d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff7931b75a924394b1af733febce0834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:37.663969 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff56618-d8f5-4622-a4f9-5a37d01cab9d service cinder] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:37.664303 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff56618-d8f5-4622-a4f9-5a37d01cab9d service cinder] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:37.665085 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff56618-d8f5-4622-a4f9-5a37d01cab9d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:37.667057 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-9ff56618-d8f5-4622-a4f9-5a37d01cab9d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:25:37.732278 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff56618-d8f5-4622-a4f9-5a37d01cab9d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:37.736131 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff56618-d8f5-4622-a4f9-5a37d01cab9d service cinder] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:37.740213 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:25:37.748032 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 952/1908] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:25:37 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:25:40.106082 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=60,keystone:INSERT=2 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:25:41.498204 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33eefa3d-2624-4586-ac85-d0112f38f462 service cinder] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:41.506835 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33eefa3d-2624-4586-ac85-d0112f38f462 service cinder] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:41.511090 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc92941-7772-4f46-a5f1-8a75fb252575 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff7931b75a924394b1af733febce0834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:41.511727 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc92941-7772-4f46-a5f1-8a75fb252575 service cinder] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:41.512036 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc92941-7772-4f46-a5f1-8a75fb252575 service cinder] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:41.512346 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc92941-7772-4f46-a5f1-8a75fb252575 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:41.521351 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc92941-7772-4f46-a5f1-8a75fb252575 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:41.522754 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc92941-7772-4f46-a5f1-8a75fb252575 service cinder] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:41.532329 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 957/1909] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:25:41 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:25:42.418031 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff56618-d8f5-4622-a4f9-5a37d01cab9d service cinder] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:42.430894 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff56618-d8f5-4622-a4f9-5a37d01cab9d service cinder] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:42.434714 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0e226c-233c-4e90-a6ab-b149547d4237 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff7931b75a924394b1af733febce0834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:42.435366 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0e226c-233c-4e90-a6ab-b149547d4237 service cinder] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:42.435696 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0e226c-233c-4e90-a6ab-b149547d4237 service cinder] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:42.436042 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0e226c-233c-4e90-a6ab-b149547d4237 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:42.437835 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-8f0e226c-233c-4e90-a6ab-b149547d4237 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:25:42.625643 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0e226c-233c-4e90-a6ab-b149547d4237 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:42.627073 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0e226c-233c-4e90-a6ab-b149547d4237 service cinder] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:42.636700 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 953/1910] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:25:42 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:25:42.842263 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc92941-7772-4f46-a5f1-8a75fb252575 service cinder] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:42.905872 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc92941-7772-4f46-a5f1-8a75fb252575 service cinder] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:42.910985 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca38ad6-7728-464c-98c6-ee09e111e875 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:42.911917 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca38ad6-7728-464c-98c6-ee09e111e875 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:42.912194 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca38ad6-7728-464c-98c6-ee09e111e875 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:42.912480 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca38ad6-7728-464c-98c6-ee09e111e875 admin admin] PATH_INFO: `/v3/users/9aa4f531c61548b8ad0a1761e78d3434` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:42.913701 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca38ad6-7728-464c-98c6-ee09e111e875 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9aa4f531c61548b8ad0a1761e78d3434)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:42.916294 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca38ad6-7728-464c-98c6-ee09e111e875 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:43.183782 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-cca38ad6-7728-464c-98c6-ee09e111e875 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9aa4f531c61548b8ad0a1761e78d3434'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:43.184450 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-cca38ad6-7728-464c-98c6-ee09e111e875 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9aa4f531c61548b8ad0a1761e78d3434'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:43.239193 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-cca38ad6-7728-464c-98c6-ee09e111e875 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9aa4f531c61548b8ad0a1761e78d3434'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:43.246762 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-cca38ad6-7728-464c-98c6-ee09e111e875 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9aa4f531c61548b8ad0a1761e78d3434'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:43.247547 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-cca38ad6-7728-464c-98c6-ee09e111e875 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9aa4f531c61548b8ad0a1761e78d3434'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:43.266826 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 958/1911] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jun 29 11:25:42 2022] DELETE /identity/v3/users/9aa4f531c61548b8ad0a1761e78d3434 => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:25:43.271445 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d14e6e5-db37-4266-b1dc-6e0a80ae5ce7 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:43.271730 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d14e6e5-db37-4266-b1dc-6e0a80ae5ce7 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:43.272159 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d14e6e5-db37-4266-b1dc-6e0a80ae5ce7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:43.290321 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-4d14e6e5-db37-4266-b1dc-6e0a80ae5ce7 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:25:43.291517 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-4d14e6e5-db37-4266-b1dc-6e0a80ae5ce7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:25:43.366607 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 954/1912] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:25:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:25:43.376650 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca38ad6-7728-464c-98c6-ee09e111e875 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:43.378064 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-cca38ad6-7728-464c-98c6-ee09e111e875 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:25:43.453025 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca38ad6-7728-464c-98c6-ee09e111e875 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:43.469603 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d27c38-bad2-4fb5-bda8-5c59cd91e009 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f4bb9f989e4024a5da28dced646b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:43.470274 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d27c38-bad2-4fb5-bda8-5c59cd91e009 service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:43.470554 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d27c38-bad2-4fb5-bda8-5c59cd91e009 service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:43.470824 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d27c38-bad2-4fb5-bda8-5c59cd91e009 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:43.481750 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d27c38-bad2-4fb5-bda8-5c59cd91e009 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:43.483398 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d27c38-bad2-4fb5-bda8-5c59cd91e009 service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:43.494165 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 959/1913] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:25:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:25:44.246694 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d14e6e5-db37-4266-b1dc-6e0a80ae5ce7 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:44.248283 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-4d14e6e5-db37-4266-b1dc-6e0a80ae5ce7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:25:44.266954 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d14e6e5-db37-4266-b1dc-6e0a80ae5ce7 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:44.272218 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dca39c-6fe4-4c82-8d4b-996f6ccdb6a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:44.281170 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dca39c-6fe4-4c82-8d4b-996f6ccdb6a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:44.281466 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dca39c-6fe4-4c82-8d4b-996f6ccdb6a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:44.281771 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dca39c-6fe4-4c82-8d4b-996f6ccdb6a0 admin admin] PATH_INFO: `/v3/projects/79c47ea37e4b4d3492b075b5a061a7e8` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:44.284319 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89dca39c-6fe4-4c82-8d4b-996f6ccdb6a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=79c47ea37e4b4d3492b075b5a061a7e8)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:44.286329 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89dca39c-6fe4-4c82-8d4b-996f6ccdb6a0 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:44.393535 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-89dca39c-6fe4-4c82-8d4b-996f6ccdb6a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '79c47ea37e4b4d3492b075b5a061a7e8', 'request_id': 'req-89dca39c-6fe4-4c82-8d4b-996f6ccdb6a0'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:44.462746 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-89dca39c-6fe4-4c82-8d4b-996f6ccdb6a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '79c47ea37e4b4d3492b075b5a061a7e8', 'request_id': 'req-89dca39c-6fe4-4c82-8d4b-996f6ccdb6a0'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:44.470173 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-89dca39c-6fe4-4c82-8d4b-996f6ccdb6a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '79c47ea37e4b4d3492b075b5a061a7e8', 'request_id': 'req-89dca39c-6fe4-4c82-8d4b-996f6ccdb6a0'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:44.479586 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d27c38-bad2-4fb5-bda8-5c59cd91e009 service neutron] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:44.481181 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-f0d27c38-bad2-4fb5-bda8-5c59cd91e009 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:25:44.481308 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-89dca39c-6fe4-4c82-8d4b-996f6ccdb6a0 admin admin] The token cache is being invalidate because project 79c47ea37e4b4d3492b075b5a061a7e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 11:25:44.481555 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-89dca39c-6fe4-4c82-8d4b-996f6ccdb6a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:44.482243 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-89dca39c-6fe4-4c82-8d4b-996f6ccdb6a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:44.483859 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 955/1914] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jun 29 11:25:44 2022] DELETE /identity/v3/projects/79c47ea37e4b4d3492b075b5a061a7e8 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:25:44.487580 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dca39c-6fe4-4c82-8d4b-996f6ccdb6a0 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:44.488928 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-89dca39c-6fe4-4c82-8d4b-996f6ccdb6a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:25:44.494580 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d27c38-bad2-4fb5-bda8-5c59cd91e009 service neutron] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:44.499429 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:25:44.507201 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:25:44.551136 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49c14bb-fb44-4b29-8ce9-3bb3672c7580 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:44.552072 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49c14bb-fb44-4b29-8ce9-3bb3672c7580 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:44.552447 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49c14bb-fb44-4b29-8ce9-3bb3672c7580 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:44.552732 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49c14bb-fb44-4b29-8ce9-3bb3672c7580 admin admin] PATH_INFO: `/v3/users/066ab897f94045da8d6aae855560bf4f` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:44.553955 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49c14bb-fb44-4b29-8ce9-3bb3672c7580 admin admin] RBAC: Authorizing `identity:delete_user(user_id=066ab897f94045da8d6aae855560bf4f)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:44.554262 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dca39c-6fe4-4c82-8d4b-996f6ccdb6a0 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:44.556810 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49c14bb-fb44-4b29-8ce9-3bb3672c7580 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:44.560843 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9986c9-d4b8-4929-b37f-010022b83c18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:44.561795 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9986c9-d4b8-4929-b37f-010022b83c18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:44.562148 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9986c9-d4b8-4929-b37f-010022b83c18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:44.562436 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9986c9-d4b8-4929-b37f-010022b83c18 admin admin] PATH_INFO: `/v3/users/a46cd446b7da474c8602395140bbf413` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:44.563643 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9986c9-d4b8-4929-b37f-010022b83c18 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a46cd446b7da474c8602395140bbf413)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:44.565588 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9986c9-d4b8-4929-b37f-010022b83c18 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:44.784607 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-e49c14bb-fb44-4b29-8ce9-3bb3672c7580 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '066ab897f94045da8d6aae855560bf4f'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:44.785202 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-e49c14bb-fb44-4b29-8ce9-3bb3672c7580 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '066ab897f94045da8d6aae855560bf4f'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:44.812648 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-2d9986c9-d4b8-4929-b37f-010022b83c18 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a46cd446b7da474c8602395140bbf413'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:44.848042 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-e49c14bb-fb44-4b29-8ce9-3bb3672c7580 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '066ab897f94045da8d6aae855560bf4f'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:44.855104 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-e49c14bb-fb44-4b29-8ce9-3bb3672c7580 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '066ab897f94045da8d6aae855560bf4f'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:44.855731 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-e49c14bb-fb44-4b29-8ce9-3bb3672c7580 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '066ab897f94045da8d6aae855560bf4f'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:44.873040 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 960/1915] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jun 29 11:25:44 2022] DELETE /identity/v3/users/066ab897f94045da8d6aae855560bf4f => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 11:25:44.877387 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a1c7a9-73e1-48b6-8adf-4ec45157af9c None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:44.877675 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a1c7a9-73e1-48b6-8adf-4ec45157af9c None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:44.877950 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a1c7a9-73e1-48b6-8adf-4ec45157af9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:44.895513 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-e8a1c7a9-73e1-48b6-8adf-4ec45157af9c None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:25:44.896737 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-e8a1c7a9-73e1-48b6-8adf-4ec45157af9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:25:44.902801 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-2d9986c9-d4b8-4929-b37f-010022b83c18 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a46cd446b7da474c8602395140bbf413'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:44.903549 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-2d9986c9-d4b8-4929-b37f-010022b83c18 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a46cd446b7da474c8602395140bbf413'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:44.934930 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-2d9986c9-d4b8-4929-b37f-010022b83c18 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a46cd446b7da474c8602395140bbf413'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:44.936001 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-2d9986c9-d4b8-4929-b37f-010022b83c18 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a46cd446b7da474c8602395140bbf413'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:44.952544 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 956/1916] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jun 29 11:25:44 2022] DELETE /identity/v3/users/a46cd446b7da474c8602395140bbf413 => generated 0 bytes in 465 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 11:25:44.966066 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 961/1917] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:25:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 11:25:44.975535 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9986c9-d4b8-4929-b37f-010022b83c18 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:44.977323 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-2d9986c9-d4b8-4929-b37f-010022b83c18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:25:45.042778 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9986c9-d4b8-4929-b37f-010022b83c18 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:45.046391 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1eaaa00-1a5b-4d38-8f81-0de51c7939f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f4bb9f989e4024a5da28dced646b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:45.047245 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eaaa00-1a5b-4d38-8f81-0de51c7939f2 service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:45.047561 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eaaa00-1a5b-4d38-8f81-0de51c7939f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:45.047866 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eaaa00-1a5b-4d38-8f81-0de51c7939f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:45.095164 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:25:45.108398 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1eaaa00-1a5b-4d38-8f81-0de51c7939f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:45.109844 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1eaaa00-1a5b-4d38-8f81-0de51c7939f2 service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:45.120154 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 957/1918] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:25:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:25:45.400277 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a1c7a9-73e1-48b6-8adf-4ec45157af9c None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:45.401923 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-e8a1c7a9-73e1-48b6-8adf-4ec45157af9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:25:45.424770 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a1c7a9-73e1-48b6-8adf-4ec45157af9c None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:45.427997 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da62eacf-5535-4867-9cb0-3ff5c26272b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:45.429963 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da62eacf-5535-4867-9cb0-3ff5c26272b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:45.431166 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da62eacf-5535-4867-9cb0-3ff5c26272b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:45.432473 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da62eacf-5535-4867-9cb0-3ff5c26272b2 admin admin] PATH_INFO: `/v3/projects/e932b5a8272d4a579a33df0c7d2012ba` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:45.433983 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da62eacf-5535-4867-9cb0-3ff5c26272b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e932b5a8272d4a579a33df0c7d2012ba)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:45.437578 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da62eacf-5535-4867-9cb0-3ff5c26272b2 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:45.473537 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:25:45.554033 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-da62eacf-5535-4867-9cb0-3ff5c26272b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e932b5a8272d4a579a33df0c7d2012ba', 'request_id': 'req-da62eacf-5535-4867-9cb0-3ff5c26272b2'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:45.560462 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-da62eacf-5535-4867-9cb0-3ff5c26272b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e932b5a8272d4a579a33df0c7d2012ba', 'request_id': 'req-da62eacf-5535-4867-9cb0-3ff5c26272b2'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:45.566276 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-da62eacf-5535-4867-9cb0-3ff5c26272b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e932b5a8272d4a579a33df0c7d2012ba', 'request_id': 'req-da62eacf-5535-4867-9cb0-3ff5c26272b2'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:45.612169 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1eaaa00-1a5b-4d38-8f81-0de51c7939f2 service neutron] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:45.613632 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-f1eaaa00-1a5b-4d38-8f81-0de51c7939f2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:25:45.628399 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1eaaa00-1a5b-4d38-8f81-0de51c7939f2 service neutron] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:45.636335 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-da62eacf-5535-4867-9cb0-3ff5c26272b2 admin admin] The token cache is being invalidate because project e932b5a8272d4a579a33df0c7d2012ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 11:25:45.636627 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-da62eacf-5535-4867-9cb0-3ff5c26272b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:45.638618 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-da62eacf-5535-4867-9cb0-3ff5c26272b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:45.641228 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 962/1919] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jun 29 11:25:45 2022] DELETE /identity/v3/projects/e932b5a8272d4a579a33df0c7d2012ba => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:25:45.645402 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da62eacf-5535-4867-9cb0-3ff5c26272b2 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:45.646947 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-da62eacf-5535-4867-9cb0-3ff5c26272b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:25:45.688774 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b91e24-6764-480e-a0f9-5295cf425dbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:45.689717 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b91e24-6764-480e-a0f9-5295cf425dbb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:45.689997 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b91e24-6764-480e-a0f9-5295cf425dbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:45.690274 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b91e24-6764-480e-a0f9-5295cf425dbb admin admin] PATH_INFO: `/v3/projects/d5024ac6c5054182bf1d15a59df3f991` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:45.691456 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b91e24-6764-480e-a0f9-5295cf425dbb admin admin] RBAC: Authorizing `identity:delete_project(project_id=d5024ac6c5054182bf1d15a59df3f991)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:45.693383 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b91e24-6764-480e-a0f9-5295cf425dbb admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:45.712847 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da62eacf-5535-4867-9cb0-3ff5c26272b2 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:45.717603 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b42678-7a9c-43b8-b080-22f9b5a0b161 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:45.718487 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b42678-7a9c-43b8-b080-22f9b5a0b161 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:45.718771 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b42678-7a9c-43b8-b080-22f9b5a0b161 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:45.719305 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b42678-7a9c-43b8-b080-22f9b5a0b161 admin admin] PATH_INFO: `/v3/users/0347a1774de84f9e943d0e3297b2f46a` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:45.720617 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b42678-7a9c-43b8-b080-22f9b5a0b161 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0347a1774de84f9e943d0e3297b2f46a)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:45.722696 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b42678-7a9c-43b8-b080-22f9b5a0b161 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:45.799927 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-48b91e24-6764-480e-a0f9-5295cf425dbb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd5024ac6c5054182bf1d15a59df3f991', 'request_id': 'req-48b91e24-6764-480e-a0f9-5295cf425dbb'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:45.859499 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-48b91e24-6764-480e-a0f9-5295cf425dbb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd5024ac6c5054182bf1d15a59df3f991', 'request_id': 'req-48b91e24-6764-480e-a0f9-5295cf425dbb'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:45.866021 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-48b91e24-6764-480e-a0f9-5295cf425dbb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd5024ac6c5054182bf1d15a59df3f991', 'request_id': 'req-48b91e24-6764-480e-a0f9-5295cf425dbb'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:45.876755 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-48b91e24-6764-480e-a0f9-5295cf425dbb admin admin] The token cache is being invalidate because project d5024ac6c5054182bf1d15a59df3f991 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 11:25:45.877051 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-48b91e24-6764-480e-a0f9-5295cf425dbb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:45.877694 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-48b91e24-6764-480e-a0f9-5295cf425dbb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:45.879505 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 958/1920] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jun 29 11:25:45 2022] DELETE /identity/v3/projects/d5024ac6c5054182bf1d15a59df3f991 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 11:25:45.883053 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b91e24-6764-480e-a0f9-5295cf425dbb admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:45.884491 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-48b91e24-6764-480e-a0f9-5295cf425dbb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:25:45.896286 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-22b42678-7a9c-43b8-b080-22f9b5a0b161 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0347a1774de84f9e943d0e3297b2f46a'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:45.896959 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-22b42678-7a9c-43b8-b080-22f9b5a0b161 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0347a1774de84f9e943d0e3297b2f46a'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:45.952994 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b91e24-6764-480e-a0f9-5295cf425dbb admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:45.957746 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d305dc-0d10-4f4e-abd9-9a5bc7f504c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:45.958614 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d305dc-0d10-4f4e-abd9-9a5bc7f504c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:45.958900 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d305dc-0d10-4f4e-abd9-9a5bc7f504c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:45.959174 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d305dc-0d10-4f4e-abd9-9a5bc7f504c2 admin admin] PATH_INFO: `/v3/users/73ec23342fcd4e489cc62bf3233b7dab` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:45.960407 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d305dc-0d10-4f4e-abd9-9a5bc7f504c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=73ec23342fcd4e489cc62bf3233b7dab)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:45.962861 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d305dc-0d10-4f4e-abd9-9a5bc7f504c2 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:45.979051 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:25:45.994069 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-22b42678-7a9c-43b8-b080-22f9b5a0b161 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0347a1774de84f9e943d0e3297b2f46a'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:46.002203 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-22b42678-7a9c-43b8-b080-22f9b5a0b161 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0347a1774de84f9e943d0e3297b2f46a'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:46.002864 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-22b42678-7a9c-43b8-b080-22f9b5a0b161 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0347a1774de84f9e943d0e3297b2f46a'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:46.020484 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 963/1921] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jun 29 11:25:45 2022] DELETE /identity/v3/users/0347a1774de84f9e943d0e3297b2f46a => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 11:25:46.144888 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-f0d305dc-0d10-4f4e-abd9-9a5bc7f504c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '73ec23342fcd4e489cc62bf3233b7dab'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:46.216954 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-f0d305dc-0d10-4f4e-abd9-9a5bc7f504c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '73ec23342fcd4e489cc62bf3233b7dab'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:46.217707 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-f0d305dc-0d10-4f4e-abd9-9a5bc7f504c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '73ec23342fcd4e489cc62bf3233b7dab'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:46.232853 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-f0d305dc-0d10-4f4e-abd9-9a5bc7f504c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '73ec23342fcd4e489cc62bf3233b7dab'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:46.233634 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-f0d305dc-0d10-4f4e-abd9-9a5bc7f504c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '73ec23342fcd4e489cc62bf3233b7dab'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:46.246340 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 959/1922] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jun 29 11:25:45 2022] DELETE /identity/v3/users/73ec23342fcd4e489cc62bf3233b7dab => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 11:25:46.468087 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b42678-7a9c-43b8-b080-22f9b5a0b161 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:46.469999 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-22b42678-7a9c-43b8-b080-22f9b5a0b161 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:25:46.528556 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:25:46.565662 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b42678-7a9c-43b8-b080-22f9b5a0b161 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:46.627954 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b04f5a-8e88-41f2-8581-eabebd41b772 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:46.627954 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b04f5a-8e88-41f2-8581-eabebd41b772 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:46.627954 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b04f5a-8e88-41f2-8581-eabebd41b772 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:46.627954 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b04f5a-8e88-41f2-8581-eabebd41b772 admin admin] PATH_INFO: `/v3/projects/a6dc7134a91f41ac90807a80195bbaf7` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:46.628327 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b04f5a-8e88-41f2-8581-eabebd41b772 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a6dc7134a91f41ac90807a80195bbaf7)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:46.629501 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b04f5a-8e88-41f2-8581-eabebd41b772 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:46.750594 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d305dc-0d10-4f4e-abd9-9a5bc7f504c2 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:46.752244 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-f0d305dc-0d10-4f4e-abd9-9a5bc7f504c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:25:46.769585 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-53b04f5a-8e88-41f2-8581-eabebd41b772 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a6dc7134a91f41ac90807a80195bbaf7', 'request_id': 'req-53b04f5a-8e88-41f2-8581-eabebd41b772'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:46.775358 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d305dc-0d10-4f4e-abd9-9a5bc7f504c2 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:46.776575 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-53b04f5a-8e88-41f2-8581-eabebd41b772 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a6dc7134a91f41ac90807a80195bbaf7', 'request_id': 'req-53b04f5a-8e88-41f2-8581-eabebd41b772'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:46.783660 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-53b04f5a-8e88-41f2-8581-eabebd41b772 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a6dc7134a91f41ac90807a80195bbaf7', 'request_id': 'req-53b04f5a-8e88-41f2-8581-eabebd41b772'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:46.840775 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e291763e-35ff-42a0-b7cf-747824d948f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:46.841628 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e291763e-35ff-42a0-b7cf-747824d948f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:46.841906 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e291763e-35ff-42a0-b7cf-747824d948f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:46.842176 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e291763e-35ff-42a0-b7cf-747824d948f7 admin admin] PATH_INFO: `/v3/projects/cd38f9511584413fac2861e2c0b8afe5` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:46.843345 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e291763e-35ff-42a0-b7cf-747824d948f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd38f9511584413fac2861e2c0b8afe5)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:46.845531 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e291763e-35ff-42a0-b7cf-747824d948f7 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:46.856595 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-53b04f5a-8e88-41f2-8581-eabebd41b772 admin admin] The token cache is being invalidate because project a6dc7134a91f41ac90807a80195bbaf7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 11:25:46.856896 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-53b04f5a-8e88-41f2-8581-eabebd41b772 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:46.857536 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-53b04f5a-8e88-41f2-8581-eabebd41b772 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:46.859129 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 964/1923] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jun 29 11:25:46 2022] DELETE /identity/v3/projects/a6dc7134a91f41ac90807a80195bbaf7 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:25:46.864880 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f28f3f-3a10-482d-a3d5-10533b762394 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:46.865174 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f28f3f-3a10-482d-a3d5-10533b762394 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:46.865459 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f28f3f-3a10-482d-a3d5-10533b762394 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:46.884025 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-24f28f3f-3a10-482d-a3d5-10533b762394 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:25:46.885232 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-24f28f3f-3a10-482d-a3d5-10533b762394 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:25:46.893147 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 965/1924] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:25:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:25:46.900918 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c746ab4a-56e6-4bb6-97d9-fc8356b49f76 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:46.901208 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c746ab4a-56e6-4bb6-97d9-fc8356b49f76 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:46.901487 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c746ab4a-56e6-4bb6-97d9-fc8356b49f76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:46.919474 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-c746ab4a-56e6-4bb6-97d9-fc8356b49f76 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:25:46.920705 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-c746ab4a-56e6-4bb6-97d9-fc8356b49f76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:25:46.928655 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 966/1925] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:25:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 11:25:46.933717 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c746ab4a-56e6-4bb6-97d9-fc8356b49f76 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:46.942719 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c746ab4a-56e6-4bb6-97d9-fc8356b49f76 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:46.952213 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-e291763e-35ff-42a0-b7cf-747824d948f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd38f9511584413fac2861e2c0b8afe5', 'request_id': 'req-e291763e-35ff-42a0-b7cf-747824d948f7'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:46.991685 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:25:46.998353 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa665df-a5a5-4ecc-8e76-c44a4acbe1bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:46.999012 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa665df-a5a5-4ecc-8e76-c44a4acbe1bf admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:46.999295 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa665df-a5a5-4ecc-8e76-c44a4acbe1bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:46.999565 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa665df-a5a5-4ecc-8e76-c44a4acbe1bf admin admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:47.000305 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa665df-a5a5-4ecc-8e76-c44a4acbe1bf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:25:47.000607 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa665df-a5a5-4ecc-8e76-c44a4acbe1bf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:47.002208 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa665df-a5a5-4ecc-8e76-c44a4acbe1bf admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:47.013269 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 967/1926] 10.222.0.44 () {58 vars in 1152 bytes} [Wed Jun 29 11:25:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:25:47.017516 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa665df-a5a5-4ecc-8e76-c44a4acbe1bf admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:47.027091 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa665df-a5a5-4ecc-8e76-c44a4acbe1bf admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:47.031822 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f651fe4-e6bd-4180-b859-ee68a99ee501 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:47.032490 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f651fe4-e6bd-4180-b859-ee68a99ee501 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:47.032771 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f651fe4-e6bd-4180-b859-ee68a99ee501 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:47.033046 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f651fe4-e6bd-4180-b859-ee68a99ee501 admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:47.033716 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f651fe4-e6bd-4180-b859-ee68a99ee501 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:47.035596 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f651fe4-e6bd-4180-b859-ee68a99ee501 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:47.036956 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-e291763e-35ff-42a0-b7cf-747824d948f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd38f9511584413fac2861e2c0b8afe5', 'request_id': 'req-e291763e-35ff-42a0-b7cf-747824d948f7'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:47.045137 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-e291763e-35ff-42a0-b7cf-747824d948f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd38f9511584413fac2861e2c0b8afe5', 'request_id': 'req-e291763e-35ff-42a0-b7cf-747824d948f7'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:47.056473 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-e291763e-35ff-42a0-b7cf-747824d948f7 admin admin] The token cache is being invalidate because project cd38f9511584413fac2861e2c0b8afe5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 11:25:47.056766 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-e291763e-35ff-42a0-b7cf-747824d948f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:47.057486 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-e291763e-35ff-42a0-b7cf-747824d948f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:47.059099 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 960/1927] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jun 29 11:25:46 2022] DELETE /identity/v3/projects/cd38f9511584413fac2861e2c0b8afe5 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:25:47.063120 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e291763e-35ff-42a0-b7cf-747824d948f7 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:47.064517 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-e291763e-35ff-42a0-b7cf-747824d948f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:25:47.076754 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e291763e-35ff-42a0-b7cf-747824d948f7 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:47.081409 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e70d0ec-f057-4fb9-9f85-71469a9d2383 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:47.082250 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e70d0ec-f057-4fb9-9f85-71469a9d2383 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:47.082534 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e70d0ec-f057-4fb9-9f85-71469a9d2383 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:47.082813 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e70d0ec-f057-4fb9-9f85-71469a9d2383 admin admin] PATH_INFO: `/v3/users/2281ceaf0c6b4565952b3ac145d77776` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:47.084258 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e70d0ec-f057-4fb9-9f85-71469a9d2383 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2281ceaf0c6b4565952b3ac145d77776)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:47.086162 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e70d0ec-f057-4fb9-9f85-71469a9d2383 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:47.109157 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:DELETE=18,keystone:SELECT=80,keystone:INSERT=2 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:25:47.143587 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 968/1928] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jun 29 11:25:47 2022] POST /identity/v3/projects => generated 371 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 11:25:47.147687 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f651fe4-e6bd-4180-b859-ee68a99ee501 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:47.149875 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-1f651fe4-e6bd-4180-b859-ee68a99ee501 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:25:47.235504 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f651fe4-e6bd-4180-b859-ee68a99ee501 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:47.242225 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3c43b3-59ab-4a0b-bc70-e23723b833b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:47.243108 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3c43b3-59ab-4a0b-bc70-e23723b833b6 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:47.243408 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3c43b3-59ab-4a0b-bc70-e23723b833b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:47.244452 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3c43b3-59ab-4a0b-bc70-e23723b833b6 admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:47.245338 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3c43b3-59ab-4a0b-bc70-e23723b833b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:47.247332 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3c43b3-59ab-4a0b-bc70-e23723b833b6 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:47.271251 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-1e70d0ec-f057-4fb9-9f85-71469a9d2383 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2281ceaf0c6b4565952b3ac145d77776'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:47.342522 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-1e70d0ec-f057-4fb9-9f85-71469a9d2383 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2281ceaf0c6b4565952b3ac145d77776'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:47.343669 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-1e70d0ec-f057-4fb9-9f85-71469a9d2383 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2281ceaf0c6b4565952b3ac145d77776'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:47.358252 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-1e70d0ec-f057-4fb9-9f85-71469a9d2383 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2281ceaf0c6b4565952b3ac145d77776'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:47.359216 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-1e70d0ec-f057-4fb9-9f85-71469a9d2383 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2281ceaf0c6b4565952b3ac145d77776'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:47.374731 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 961/1929] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jun 29 11:25:47 2022] DELETE /identity/v3/users/2281ceaf0c6b4565952b3ac145d77776 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 11:25:47.409001 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 969/1930] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jun 29 11:25:47 2022] POST /identity/v3/users => generated 286 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 11:25:47.412872 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e70d0ec-f057-4fb9-9f85-71469a9d2383 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:47.414483 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-1e70d0ec-f057-4fb9-9f85-71469a9d2383 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:25:47.503938 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e70d0ec-f057-4fb9-9f85-71469a9d2383 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:47.510381 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4d1b99-5ae9-44d3-9210-1fd1c082373f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:47.511066 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4d1b99-5ae9-44d3-9210-1fd1c082373f admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:47.511357 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4d1b99-5ae9-44d3-9210-1fd1c082373f admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:47.511634 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4d1b99-5ae9-44d3-9210-1fd1c082373f admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:47.512402 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4d1b99-5ae9-44d3-9210-1fd1c082373f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:47.514017 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4d1b99-5ae9-44d3-9210-1fd1c082373f admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:47.525250 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 962/1931] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:25:47 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 11:25:47.529594 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3c43b3-59ab-4a0b-bc70-e23723b833b6 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:47.540356 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3c43b3-59ab-4a0b-bc70-e23723b833b6 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:47.544842 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223a899d-86d1-4695-b502-ddeaefec6589 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:47.545490 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223a899d-86d1-4695-b502-ddeaefec6589 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:47.545771 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223a899d-86d1-4695-b502-ddeaefec6589 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:47.546044 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223a899d-86d1-4695-b502-ddeaefec6589 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:47.546854 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223a899d-86d1-4695-b502-ddeaefec6589 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:47.548516 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223a899d-86d1-4695-b502-ddeaefec6589 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:47.563067 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 970/1932] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:25:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 11:25:47.564497 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4d1b99-5ae9-44d3-9210-1fd1c082373f admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:47.575967 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4d1b99-5ae9-44d3-9210-1fd1c082373f admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:47.586352 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e21593-5953-409a-b80f-4bc328a428e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:47.586352 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e21593-5953-409a-b80f-4bc328a428e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:47.586352 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e21593-5953-409a-b80f-4bc328a428e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:47.586352 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e21593-5953-409a-b80f-4bc328a428e1 admin admin] PATH_INFO: `/v3/projects/a16cac87d3c44485ab2bdff482d33052/users/ccc29dd5c1eb45f08b0a18fb39fc93c4/roles/e1c88eebe911432cab6a07fbeb84653b` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:47.616791 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e21593-5953-409a-b80f-4bc328a428e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a16cac87d3c44485ab2bdff482d33052, user_id=ccc29dd5c1eb45f08b0a18fb39fc93c4, role_id=e1c88eebe911432cab6a07fbeb84653b)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:47.620664 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e21593-5953-409a-b80f-4bc328a428e1 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:47.697477 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 963/1933] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jun 29 11:25:47 2022] PUT /identity/v3/projects/a16cac87d3c44485ab2bdff482d33052/users/ccc29dd5c1eb45f08b0a18fb39fc93c4/roles/e1c88eebe911432cab6a07fbeb84653b => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:25:47.702845 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7141f9cb-06a7-4173-bab8-30b24805c8e2 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:47.703352 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7141f9cb-06a7-4173-bab8-30b24805c8e2 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:47.703442 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7141f9cb-06a7-4173-bab8-30b24805c8e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:47.722599 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-7141f9cb-06a7-4173-bab8-30b24805c8e2 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:25:47.724378 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-7141f9cb-06a7-4173-bab8-30b24805c8e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:25:47.792769 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 971/1934] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:25:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:25:47.802094 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e21593-5953-409a-b80f-4bc328a428e1 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:47.806360 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-42e21593-5953-409a-b80f-4bc328a428e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:25:47.863606 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e21593-5953-409a-b80f-4bc328a428e1 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:47.867215 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c608a660-8eed-439e-bbe2-0da6ff8d7e54 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f4bb9f989e4024a5da28dced646b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:47.867938 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c608a660-8eed-439e-bbe2-0da6ff8d7e54 service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:47.868220 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c608a660-8eed-439e-bbe2-0da6ff8d7e54 service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:47.868493 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c608a660-8eed-439e-bbe2-0da6ff8d7e54 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:47.878077 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c608a660-8eed-439e-bbe2-0da6ff8d7e54 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:47.879673 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c608a660-8eed-439e-bbe2-0da6ff8d7e54 service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:47.891001 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 964/1935] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:25:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:25:47.914681 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7141f9cb-06a7-4173-bab8-30b24805c8e2 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:47.916255 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-7141f9cb-06a7-4173-bab8-30b24805c8e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:25:47.931022 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7141f9cb-06a7-4173-bab8-30b24805c8e2 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:47.936284 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3995792-0ed6-4a0b-8c36-88d592854188 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:47.937207 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3995792-0ed6-4a0b-8c36-88d592854188 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:47.937490 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3995792-0ed6-4a0b-8c36-88d592854188 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:47.937785 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3995792-0ed6-4a0b-8c36-88d592854188 admin admin] PATH_INFO: `/v3/projects/ede3a053ecb54f28ba1129c148f9d6ed` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:47.938958 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3995792-0ed6-4a0b-8c36-88d592854188 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ede3a053ecb54f28ba1129c148f9d6ed)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:47.940909 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3995792-0ed6-4a0b-8c36-88d592854188 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:48.084749 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:25:48.095320 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-a3995792-0ed6-4a0b-8c36-88d592854188 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ede3a053ecb54f28ba1129c148f9d6ed', 'request_id': 'req-a3995792-0ed6-4a0b-8c36-88d592854188'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:48.101620 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-a3995792-0ed6-4a0b-8c36-88d592854188 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ede3a053ecb54f28ba1129c148f9d6ed', 'request_id': 'req-a3995792-0ed6-4a0b-8c36-88d592854188'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:48.108550 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-a3995792-0ed6-4a0b-8c36-88d592854188 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ede3a053ecb54f28ba1129c148f9d6ed', 'request_id': 'req-a3995792-0ed6-4a0b-8c36-88d592854188'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:48.185367 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-a3995792-0ed6-4a0b-8c36-88d592854188 admin admin] The token cache is being invalidate because project ede3a053ecb54f28ba1129c148f9d6ed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 11:25:48.185722 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-a3995792-0ed6-4a0b-8c36-88d592854188 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:48.186478 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-a3995792-0ed6-4a0b-8c36-88d592854188 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:25:48.188444 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 972/1936] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jun 29 11:25:47 2022] DELETE /identity/v3/projects/ede3a053ecb54f28ba1129c148f9d6ed => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:25:48.195180 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbbb451-ade7-4922-a62d-ae4e3bf0a500 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:48.195795 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbbb451-ade7-4922-a62d-ae4e3bf0a500 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:48.196359 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbbb451-ade7-4922-a62d-ae4e3bf0a500 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:48.216206 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-fcbbb451-ade7-4922-a62d-ae4e3bf0a500 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:25:48.217772 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-fcbbb451-ade7-4922-a62d-ae4e3bf0a500 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:25:48.224434 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:DELETE=8,keystone:SELECT=90,keystone:INSERT=2 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:25:48.279945 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 965/1937] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:25:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:25:48.288413 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee7e8ed-3032-4f3f-8257-1e277311794d None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:48.288705 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee7e8ed-3032-4f3f-8257-1e277311794d None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:48.288986 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee7e8ed-3032-4f3f-8257-1e277311794d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:48.308985 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-9ee7e8ed-3032-4f3f-8257-1e277311794d None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:25:48.310271 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-9ee7e8ed-3032-4f3f-8257-1e277311794d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:25:48.318284 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 973/1938] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:25:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:25:48.323038 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbbb451-ade7-4922-a62d-ae4e3bf0a500 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:48.332589 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbbb451-ade7-4922-a62d-ae4e3bf0a500 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:48.337882 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be0d2dd-98eb-4717-bc8c-6e00b4bf2649 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:48.338596 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be0d2dd-98eb-4717-bc8c-6e00b4bf2649 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:48.338880 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be0d2dd-98eb-4717-bc8c-6e00b4bf2649 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:48.339156 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be0d2dd-98eb-4717-bc8c-6e00b4bf2649 admin admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:48.339911 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be0d2dd-98eb-4717-bc8c-6e00b4bf2649 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:25:48.340219 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be0d2dd-98eb-4717-bc8c-6e00b4bf2649 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:48.344627 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be0d2dd-98eb-4717-bc8c-6e00b4bf2649 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:48.353464 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 966/1939] 10.222.0.44 () {58 vars in 1152 bytes} [Wed Jun 29 11:25:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:25:48.357859 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee7e8ed-3032-4f3f-8257-1e277311794d None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:48.367654 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee7e8ed-3032-4f3f-8257-1e277311794d None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:48.372594 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea14776-16f4-44ef-b194-4e89f25562e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:48.373279 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea14776-16f4-44ef-b194-4e89f25562e3 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:48.373583 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea14776-16f4-44ef-b194-4e89f25562e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:48.373840 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea14776-16f4-44ef-b194-4e89f25562e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:48.374541 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea14776-16f4-44ef-b194-4e89f25562e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:48.376513 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea14776-16f4-44ef-b194-4e89f25562e3 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:48.481546 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 974/1940] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jun 29 11:25:48 2022] POST /identity/v3/projects => generated 409 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 11:25:48.486612 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be0d2dd-98eb-4717-bc8c-6e00b4bf2649 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:48.522835 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be0d2dd-98eb-4717-bc8c-6e00b4bf2649 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:48.617386 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4085df53-c4ab-4a1c-a583-e64511f4e9cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:48.618093 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4085df53-c4ab-4a1c-a583-e64511f4e9cc admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:48.618378 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4085df53-c4ab-4a1c-a583-e64511f4e9cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:48.618653 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4085df53-c4ab-4a1c-a583-e64511f4e9cc admin admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:48.622662 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4085df53-c4ab-4a1c-a583-e64511f4e9cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:48.624745 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4085df53-c4ab-4a1c-a583-e64511f4e9cc admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:48.706049 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea14776-16f4-44ef-b194-4e89f25562e3 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:48.713508 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 967/1941] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jun 29 11:25:48 2022] POST /identity/v3/users => generated 305 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:25:48.721325 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea14776-16f4-44ef-b194-4e89f25562e3 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:48.733624 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9945ede7-654b-4d8f-9c70-8138cccb32bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:48.734305 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9945ede7-654b-4d8f-9c70-8138cccb32bc admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:48.734545 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9945ede7-654b-4d8f-9c70-8138cccb32bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:48.734830 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9945ede7-654b-4d8f-9c70-8138cccb32bc admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:48.735566 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9945ede7-654b-4d8f-9c70-8138cccb32bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:48.738228 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9945ede7-654b-4d8f-9c70-8138cccb32bc admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:48.755486 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 975/1942] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:25:48 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 11:25:48.757814 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4085df53-c4ab-4a1c-a583-e64511f4e9cc admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:48.771137 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4085df53-c4ab-4a1c-a583-e64511f4e9cc admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:48.776600 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6f01c0-ba0d-4e1e-9771-c0dfadbf3625 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:48.777648 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6f01c0-ba0d-4e1e-9771-c0dfadbf3625 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:48.778076 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6f01c0-ba0d-4e1e-9771-c0dfadbf3625 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:48.778505 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6f01c0-ba0d-4e1e-9771-c0dfadbf3625 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:48.779489 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6f01c0-ba0d-4e1e-9771-c0dfadbf3625 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:48.781345 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6f01c0-ba0d-4e1e-9771-c0dfadbf3625 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:48.798118 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 968/1943] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:25:48 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 11:25:48.799529 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9945ede7-654b-4d8f-9c70-8138cccb32bc admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:48.814650 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9945ede7-654b-4d8f-9c70-8138cccb32bc admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:48.820271 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1362ab5e-6a4a-4ebc-9184-d54893487ce9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:48.821957 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1362ab5e-6a4a-4ebc-9184-d54893487ce9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:48.822656 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1362ab5e-6a4a-4ebc-9184-d54893487ce9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:48.823126 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1362ab5e-6a4a-4ebc-9184-d54893487ce9 admin admin] PATH_INFO: `/v3/projects/d6e15d1877044eeaae711e2cce5fc256/users/8ae43b7540dd450e9a19eb0b2c85af8e/roles/e1c88eebe911432cab6a07fbeb84653b` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:48.842577 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1362ab5e-6a4a-4ebc-9184-d54893487ce9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6e15d1877044eeaae711e2cce5fc256, user_id=8ae43b7540dd450e9a19eb0b2c85af8e, role_id=e1c88eebe911432cab6a07fbeb84653b)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:48.846066 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1362ab5e-6a4a-4ebc-9184-d54893487ce9 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:48.944370 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 976/1944] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jun 29 11:25:48 2022] PUT /identity/v3/projects/d6e15d1877044eeaae711e2cce5fc256/users/8ae43b7540dd450e9a19eb0b2c85af8e/roles/e1c88eebe911432cab6a07fbeb84653b => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:25:48.956655 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee9a758-ab51-4733-91ef-71244537c9dd None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:48.957592 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee9a758-ab51-4733-91ef-71244537c9dd None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:48.957959 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee9a758-ab51-4733-91ef-71244537c9dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:49.001685 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-5ee9a758-ab51-4733-91ef-71244537c9dd None None] MFA Rules not processed for user `8ae43b7540dd450e9a19eb0b2c85af8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:25:49.004663 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-5ee9a758-ab51-4733-91ef-71244537c9dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:25:49.077463 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 969/1945] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:25:48 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:25:49.078347 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1362ab5e-6a4a-4ebc-9184-d54893487ce9 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:49.166071 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1362ab5e-6a4a-4ebc-9184-d54893487ce9 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:49.170976 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6baf8b-1014-49de-be19-b581f515f5e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:49.171718 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6baf8b-1014-49de-be19-b581f515f5e1 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:49.172130 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6baf8b-1014-49de-be19-b581f515f5e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:49.172485 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6baf8b-1014-49de-be19-b581f515f5e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:49.173188 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6baf8b-1014-49de-be19-b581f515f5e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:49.175099 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6baf8b-1014-49de-be19-b581f515f5e1 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:49.283963 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 977/1946] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jun 29 11:25:49 2022] POST /identity/v3/projects => generated 409 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:25:49.288008 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee9a758-ab51-4733-91ef-71244537c9dd None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:49.298642 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee9a758-ab51-4733-91ef-71244537c9dd None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:49.302243 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:25:49.355171 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ea0bd9-b00e-4c9b-b231-400c4de3fbdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:49.355884 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ea0bd9-b00e-4c9b-b231-400c4de3fbdd admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:49.356178 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ea0bd9-b00e-4c9b-b231-400c4de3fbdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:49.356460 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ea0bd9-b00e-4c9b-b231-400c4de3fbdd admin admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:49.357277 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ea0bd9-b00e-4c9b-b231-400c4de3fbdd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:49.359227 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ea0bd9-b00e-4c9b-b231-400c4de3fbdd admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:49.475114 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 970/1947] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jun 29 11:25:49 2022] POST /identity/v3/users => generated 311 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:25:49.480648 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6baf8b-1014-49de-be19-b581f515f5e1 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:49.496485 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6baf8b-1014-49de-be19-b581f515f5e1 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:49.501346 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900125bc-4c94-422d-8e93-bf8ad4780a87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:49.505805 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900125bc-4c94-422d-8e93-bf8ad4780a87 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:49.505805 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900125bc-4c94-422d-8e93-bf8ad4780a87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:49.505805 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900125bc-4c94-422d-8e93-bf8ad4780a87 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:49.505805 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900125bc-4c94-422d-8e93-bf8ad4780a87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:49.505805 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900125bc-4c94-422d-8e93-bf8ad4780a87 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:49.520656 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 978/1948] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:25:49 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 11:25:49.522407 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ea0bd9-b00e-4c9b-b231-400c4de3fbdd admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:49.534503 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ea0bd9-b00e-4c9b-b231-400c4de3fbdd admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:49.539388 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b99665a-72d2-4069-bd00-80d946ffc40d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:49.540358 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b99665a-72d2-4069-bd00-80d946ffc40d admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:49.540674 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b99665a-72d2-4069-bd00-80d946ffc40d admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:49.540952 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b99665a-72d2-4069-bd00-80d946ffc40d admin admin] PATH_INFO: `/v3/projects/781ee553cbe04cc8b7f147d6bf399ebe/users/71569f5a94df46029d94b1cd33f6211b/roles/e95c2ce63e114cc19a9dda6ad21e794f` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:49.566665 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b99665a-72d2-4069-bd00-80d946ffc40d admin admin] RBAC: Authorizing `identity:create_grant(project_id=781ee553cbe04cc8b7f147d6bf399ebe, user_id=71569f5a94df46029d94b1cd33f6211b, role_id=e95c2ce63e114cc19a9dda6ad21e794f)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:49.569510 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b99665a-72d2-4069-bd00-80d946ffc40d admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:49.571937 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97a8d6d-a273-41da-8aa6-c002fc910ae1 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:49.572232 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97a8d6d-a273-41da-8aa6-c002fc910ae1 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:49.572511 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97a8d6d-a273-41da-8aa6-c002fc910ae1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:49.600053 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-b97a8d6d-a273-41da-8aa6-c002fc910ae1 None None] MFA Rules not processed for user `ccc29dd5c1eb45f08b0a18fb39fc93c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:25:49.601305 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-b97a8d6d-a273-41da-8aa6-c002fc910ae1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:25:49.634748 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:25:49.686526 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 979/1949] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:25:49 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:25:49.690908 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97a8d6d-a273-41da-8aa6-c002fc910ae1 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:49.692593 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-b97a8d6d-a273-41da-8aa6-c002fc910ae1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:25:49.709700 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97a8d6d-a273-41da-8aa6-c002fc910ae1 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:49.714839 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cb750d-3420-4577-b422-6465ed37adc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:49.715508 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cb750d-3420-4577-b422-6465ed37adc6 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:49.715797 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cb750d-3420-4577-b422-6465ed37adc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:49.716108 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cb750d-3420-4577-b422-6465ed37adc6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:49.716807 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cb750d-3420-4577-b422-6465ed37adc6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:49.718879 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cb750d-3420-4577-b422-6465ed37adc6 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:49.720516 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 971/1950] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jun 29 11:25:49 2022] PUT /identity/v3/projects/781ee553cbe04cc8b7f147d6bf399ebe/users/71569f5a94df46029d94b1cd33f6211b/roles/e95c2ce63e114cc19a9dda6ad21e794f => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 11:25:49.724456 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b99665a-72d2-4069-bd00-80d946ffc40d admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:49.799607 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 980/1951] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jun 29 11:25:49 2022] POST /identity/v3/projects => generated 373 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 11:25:49.803315 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cb750d-3420-4577-b422-6465ed37adc6 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:49.805777 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b99665a-72d2-4069-bd00-80d946ffc40d admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:49.864483 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c427b32-76a9-4659-9f04-bffc4c9e5b30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:49.865041 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c427b32-76a9-4659-9f04-bffc4c9e5b30 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:49.865328 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c427b32-76a9-4659-9f04-bffc4c9e5b30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:49.865606 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c427b32-76a9-4659-9f04-bffc4c9e5b30 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:49.866353 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c427b32-76a9-4659-9f04-bffc4c9e5b30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:49.868659 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c427b32-76a9-4659-9f04-bffc4c9e5b30 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:49.871174 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cb750d-3420-4577-b422-6465ed37adc6 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:49.877795 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de798983-acba-4853-8d09-c4d53e737544 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:49.878499 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de798983-acba-4853-8d09-c4d53e737544 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:49.878790 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de798983-acba-4853-8d09-c4d53e737544 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:49.879070 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de798983-acba-4853-8d09-c4d53e737544 admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:49.879755 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de798983-acba-4853-8d09-c4d53e737544 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:49.881705 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de798983-acba-4853-8d09-c4d53e737544 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:49.888880 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 972/1952] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:25:49 2022] GET /identity/v3/roles => generated 1420 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 11:25:49.893489 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c427b32-76a9-4659-9f04-bffc4c9e5b30 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:49.904245 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c427b32-76a9-4659-9f04-bffc4c9e5b30 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:49.910895 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299dde8c-502f-42f3-ad1c-bcb520c48c6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:49.912202 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299dde8c-502f-42f3-ad1c-bcb520c48c6b admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:49.912202 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299dde8c-502f-42f3-ad1c-bcb520c48c6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:49.912407 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299dde8c-502f-42f3-ad1c-bcb520c48c6b admin admin] PATH_INFO: `/v3/projects/781ee553cbe04cc8b7f147d6bf399ebe/users/71569f5a94df46029d94b1cd33f6211b/roles/e95c2ce63e114cc19a9dda6ad21e794f` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:49.915290 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299dde8c-502f-42f3-ad1c-bcb520c48c6b admin admin] RBAC: Authorizing `identity:create_grant(project_id=781ee553cbe04cc8b7f147d6bf399ebe, user_id=71569f5a94df46029d94b1cd33f6211b, role_id=e95c2ce63e114cc19a9dda6ad21e794f)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:49.918755 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299dde8c-502f-42f3-ad1c-bcb520c48c6b admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:49.931028 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 973/1953] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jun 29 11:25:49 2022] PUT /identity/v3/projects/781ee553cbe04cc8b7f147d6bf399ebe/users/71569f5a94df46029d94b1cd33f6211b/roles/e95c2ce63e114cc19a9dda6ad21e794f => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:25:49.941272 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58365480-46aa-4302-8d7f-d4fbd498992d None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:49.941565 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58365480-46aa-4302-8d7f-d4fbd498992d None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:49.941840 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58365480-46aa-4302-8d7f-d4fbd498992d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:49.970238 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-58365480-46aa-4302-8d7f-d4fbd498992d None None] MFA Rules not processed for user `71569f5a94df46029d94b1cd33f6211b`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:25:49.971470 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-58365480-46aa-4302-8d7f-d4fbd498992d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:25:49.995971 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 981/1954] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jun 29 11:25:49 2022] POST /identity/v3/users => generated 293 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 11:25:49.997333 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de798983-acba-4853-8d09-c4d53e737544 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:50.010657 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de798983-acba-4853-8d09-c4d53e737544 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:50.069126 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 974/1955] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:25:49 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:25:50.081654 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26db0dbb-e374-45fd-941c-86cca521a7d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:50.082555 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26db0dbb-e374-45fd-941c-86cca521a7d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:50.082861 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26db0dbb-e374-45fd-941c-86cca521a7d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:50.083139 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26db0dbb-e374-45fd-941c-86cca521a7d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:50.083891 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26db0dbb-e374-45fd-941c-86cca521a7d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:50.085882 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26db0dbb-e374-45fd-941c-86cca521a7d1 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:50.087934 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58365480-46aa-4302-8d7f-d4fbd498992d None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:50.089412 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-58365480-46aa-4302-8d7f-d4fbd498992d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:25:50.111876 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 982/1956] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:25:49 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 11:25:50.114442 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26db0dbb-e374-45fd-941c-86cca521a7d1 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:50.136581 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26db0dbb-e374-45fd-941c-86cca521a7d1 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:50.149822 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28b5f8b-be6a-4c65-b7a3-ca99171197e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:50.152058 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28b5f8b-be6a-4c65-b7a3-ca99171197e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:50.152633 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28b5f8b-be6a-4c65-b7a3-ca99171197e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:50.153597 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28b5f8b-be6a-4c65-b7a3-ca99171197e7 admin admin] PATH_INFO: `/v3/projects/bfd9203dfff74f1f864bfa6d10e34103/users/27568bd41dba421f956eee06b3d3d0d3/roles/e95c2ce63e114cc19a9dda6ad21e794f` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:50.172977 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58365480-46aa-4302-8d7f-d4fbd498992d None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:50.174015 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28b5f8b-be6a-4c65-b7a3-ca99171197e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bfd9203dfff74f1f864bfa6d10e34103, user_id=27568bd41dba421f956eee06b3d3d0d3, role_id=e95c2ce63e114cc19a9dda6ad21e794f)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:50.177855 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5227e4a4-3fb3-407e-86d8-0ef8726d675b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff7931b75a924394b1af733febce0834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:50.178961 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5227e4a4-3fb3-407e-86d8-0ef8726d675b service cinder] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:50.178961 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5227e4a4-3fb3-407e-86d8-0ef8726d675b service cinder] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:50.179350 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5227e4a4-3fb3-407e-86d8-0ef8726d675b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:50.180808 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28b5f8b-be6a-4c65-b7a3-ca99171197e7 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:50.189917 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5227e4a4-3fb3-407e-86d8-0ef8726d675b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:50.191367 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5227e4a4-3fb3-407e-86d8-0ef8726d675b service cinder] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:50.201548 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 975/1957] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:25:50 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 11:25:50.295503 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 983/1958] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jun 29 11:25:50 2022] PUT /identity/v3/projects/bfd9203dfff74f1f864bfa6d10e34103/users/27568bd41dba421f956eee06b3d3d0d3/roles/e95c2ce63e114cc19a9dda6ad21e794f => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:25:50.298412 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5227e4a4-3fb3-407e-86d8-0ef8726d675b service cinder] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:50.323978 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5227e4a4-3fb3-407e-86d8-0ef8726d675b service cinder] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:50.325746 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:25:50.390900 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62c491d-f709-42a5-a2d8-d62db560f9a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:50.391488 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62c491d-f709-42a5-a2d8-d62db560f9a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:50.391768 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62c491d-f709-42a5-a2d8-d62db560f9a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:50.392079 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62c491d-f709-42a5-a2d8-d62db560f9a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:50.392804 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62c491d-f709-42a5-a2d8-d62db560f9a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:50.394428 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62c491d-f709-42a5-a2d8-d62db560f9a3 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:50.406346 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 976/1959] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:25:50 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 11:25:50.410346 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28b5f8b-be6a-4c65-b7a3-ca99171197e7 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:50.421633 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28b5f8b-be6a-4c65-b7a3-ca99171197e7 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:50.426277 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e1c044-f788-49bc-8947-90a9efa7cc0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:50.427390 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e1c044-f788-49bc-8947-90a9efa7cc0b admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:50.427489 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e1c044-f788-49bc-8947-90a9efa7cc0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:50.427768 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e1c044-f788-49bc-8947-90a9efa7cc0b admin admin] PATH_INFO: `/v3/projects/bfd9203dfff74f1f864bfa6d10e34103/users/27568bd41dba421f956eee06b3d3d0d3/roles/e95c2ce63e114cc19a9dda6ad21e794f` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:50.429759 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e1c044-f788-49bc-8947-90a9efa7cc0b admin admin] RBAC: Authorizing `identity:create_grant(project_id=bfd9203dfff74f1f864bfa6d10e34103, user_id=27568bd41dba421f956eee06b3d3d0d3, role_id=e95c2ce63e114cc19a9dda6ad21e794f)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:50.432544 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e1c044-f788-49bc-8947-90a9efa7cc0b admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:50.443468 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 984/1960] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jun 29 11:25:50 2022] PUT /identity/v3/projects/bfd9203dfff74f1f864bfa6d10e34103/users/27568bd41dba421f956eee06b3d3d0d3/roles/e95c2ce63e114cc19a9dda6ad21e794f => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:25:50.834300 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62c491d-f709-42a5-a2d8-d62db560f9a3 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:50.836045 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-b62c491d-f709-42a5-a2d8-d62db560f9a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:25:50.892955 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62c491d-f709-42a5-a2d8-d62db560f9a3 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:50.965330 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1722adf0-c3f4-40f7-8d85-25574d7baf04 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff7931b75a924394b1af733febce0834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:50.966024 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1722adf0-c3f4-40f7-8d85-25574d7baf04 service cinder] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:50.966477 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1722adf0-c3f4-40f7-8d85-25574d7baf04 service cinder] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:50.970826 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1722adf0-c3f4-40f7-8d85-25574d7baf04 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:51.028690 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1722adf0-c3f4-40f7-8d85-25574d7baf04 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:51.030368 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1722adf0-c3f4-40f7-8d85-25574d7baf04 service cinder] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:51.040419 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 977/1961] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:25:50 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:25:51.340305 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:25:51.834467 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:25:51.855305 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b6f474-d26d-4bcc-9721-3e663a2e7372 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:51.855671 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b6f474-d26d-4bcc-9721-3e663a2e7372 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:51.856095 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b6f474-d26d-4bcc-9721-3e663a2e7372 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:51.885869 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-68b6f474-d26d-4bcc-9721-3e663a2e7372 None None] MFA Rules not processed for user `27568bd41dba421f956eee06b3d3d0d3`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:25:51.887317 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-68b6f474-d26d-4bcc-9721-3e663a2e7372 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:25:51.974446 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 985/1962] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:25:51 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 11:25:51.986059 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1722adf0-c3f4-40f7-8d85-25574d7baf04 service cinder] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:51.996252 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1722adf0-c3f4-40f7-8d85-25574d7baf04 service cinder] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:52.000013 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea7f801-75ef-4975-aaae-095f7db67a3d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff7931b75a924394b1af733febce0834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:52.000651 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea7f801-75ef-4975-aaae-095f7db67a3d service cinder] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:52.000931 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea7f801-75ef-4975-aaae-095f7db67a3d service cinder] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:52.001204 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea7f801-75ef-4975-aaae-095f7db67a3d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:52.060450 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea7f801-75ef-4975-aaae-095f7db67a3d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:52.061879 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea7f801-75ef-4975-aaae-095f7db67a3d service cinder] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:52.071774 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 978/1963] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:25:51 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:25:59.051699 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b6f474-d26d-4bcc-9721-3e663a2e7372 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:25:59.059386 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b6f474-d26d-4bcc-9721-3e663a2e7372 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:25:59.062917 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d2e1a9-25f9-459d-b979-bb030fd2b0b7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff7931b75a924394b1af733febce0834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:25:59.063556 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d2e1a9-25f9-459d-b979-bb030fd2b0b7 service cinder] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:25:59.063853 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d2e1a9-25f9-459d-b979-bb030fd2b0b7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:25:59.064174 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d2e1a9-25f9-459d-b979-bb030fd2b0b7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:25:59.074231 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d2e1a9-25f9-459d-b979-bb030fd2b0b7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:25:59.076212 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d2e1a9-25f9-459d-b979-bb030fd2b0b7 service cinder] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:25:59.088338 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 986/1964] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:25:59 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:26:02.711370 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=84 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:26:09.082447 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=42 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:26:18.676113 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea7f801-75ef-4975-aaae-095f7db67a3d service cinder] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:18.677544 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-2ea7f801-75ef-4975-aaae-095f7db67a3d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:26:18.745475 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea7f801-75ef-4975-aaae-095f7db67a3d service cinder] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:18.750283 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888de4ae-1238-47fe-8aeb-3fa9054f2010 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:18.751098 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888de4ae-1238-47fe-8aeb-3fa9054f2010 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:18.751414 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888de4ae-1238-47fe-8aeb-3fa9054f2010 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:18.751716 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888de4ae-1238-47fe-8aeb-3fa9054f2010 admin admin] PATH_INFO: `/v3/users/5a23e00456fd430f8656b0ca4e6e8aac` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:18.752929 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888de4ae-1238-47fe-8aeb-3fa9054f2010 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a23e00456fd430f8656b0ca4e6e8aac)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:18.754958 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888de4ae-1238-47fe-8aeb-3fa9054f2010 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:18.917239 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-888de4ae-1238-47fe-8aeb-3fa9054f2010 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a23e00456fd430f8656b0ca4e6e8aac'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:19.007278 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-888de4ae-1238-47fe-8aeb-3fa9054f2010 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a23e00456fd430f8656b0ca4e6e8aac'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:19.007970 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-888de4ae-1238-47fe-8aeb-3fa9054f2010 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a23e00456fd430f8656b0ca4e6e8aac'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:19.025182 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-888de4ae-1238-47fe-8aeb-3fa9054f2010 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a23e00456fd430f8656b0ca4e6e8aac'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:19.025923 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-888de4ae-1238-47fe-8aeb-3fa9054f2010 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a23e00456fd430f8656b0ca4e6e8aac'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:19.043175 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 979/1965] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jun 29 11:26:18 2022] DELETE /identity/v3/users/5a23e00456fd430f8656b0ca4e6e8aac => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:26:19.044420 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc9262e-9339-4306-9b6f-b19ea3a4e359 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:19.047440 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc9262e-9339-4306-9b6f-b19ea3a4e359 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:19.047440 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc9262e-9339-4306-9b6f-b19ea3a4e359 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:19.069866 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-7bc9262e-9339-4306-9b6f-b19ea3a4e359 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:26:19.071149 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-7bc9262e-9339-4306-9b6f-b19ea3a4e359 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:26:19.131910 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 987/1966] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:26:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:26:19.144886 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888de4ae-1238-47fe-8aeb-3fa9054f2010 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:19.144886 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-888de4ae-1238-47fe-8aeb-3fa9054f2010 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:26:19.208180 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888de4ae-1238-47fe-8aeb-3fa9054f2010 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:19.211916 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c256dc-0693-4986-b815-650b12f86d30 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f4bb9f989e4024a5da28dced646b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:19.220170 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c256dc-0693-4986-b815-650b12f86d30 service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:19.220170 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c256dc-0693-4986-b815-650b12f86d30 service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:19.220170 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c256dc-0693-4986-b815-650b12f86d30 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:19.231173 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c256dc-0693-4986-b815-650b12f86d30 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:19.232651 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c256dc-0693-4986-b815-650b12f86d30 service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:19.242481 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 980/1967] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:26:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:26:19.663950 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc9262e-9339-4306-9b6f-b19ea3a4e359 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:19.668260 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-7bc9262e-9339-4306-9b6f-b19ea3a4e359 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:26:19.701102 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc9262e-9339-4306-9b6f-b19ea3a4e359 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:19.703587 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4479ac37-ee25-4026-baf9-9ac0be9ba39f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:19.704496 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4479ac37-ee25-4026-baf9-9ac0be9ba39f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:19.705010 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4479ac37-ee25-4026-baf9-9ac0be9ba39f admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:19.705489 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4479ac37-ee25-4026-baf9-9ac0be9ba39f admin admin] PATH_INFO: `/v3/projects/717a5f10aba945cb889b19f0fb0cbb8d` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:19.709385 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4479ac37-ee25-4026-baf9-9ac0be9ba39f admin admin] RBAC: Authorizing `identity:delete_project(project_id=717a5f10aba945cb889b19f0fb0cbb8d)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:19.711353 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4479ac37-ee25-4026-baf9-9ac0be9ba39f admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:19.846511 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-4479ac37-ee25-4026-baf9-9ac0be9ba39f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '717a5f10aba945cb889b19f0fb0cbb8d', 'request_id': 'req-4479ac37-ee25-4026-baf9-9ac0be9ba39f'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:19.853523 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-4479ac37-ee25-4026-baf9-9ac0be9ba39f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '717a5f10aba945cb889b19f0fb0cbb8d', 'request_id': 'req-4479ac37-ee25-4026-baf9-9ac0be9ba39f'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:19.859610 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-4479ac37-ee25-4026-baf9-9ac0be9ba39f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '717a5f10aba945cb889b19f0fb0cbb8d', 'request_id': 'req-4479ac37-ee25-4026-baf9-9ac0be9ba39f'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:19.927029 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-4479ac37-ee25-4026-baf9-9ac0be9ba39f admin admin] The token cache is being invalidate because project 717a5f10aba945cb889b19f0fb0cbb8d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 11:26:19.927304 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-4479ac37-ee25-4026-baf9-9ac0be9ba39f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:19.929576 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-4479ac37-ee25-4026-baf9-9ac0be9ba39f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:19.929680 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 988/1968] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jun 29 11:26:19 2022] DELETE /identity/v3/projects/717a5f10aba945cb889b19f0fb0cbb8d => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:26:19.933439 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c256dc-0693-4986-b815-650b12f86d30 service neutron] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:19.943167 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c256dc-0693-4986-b815-650b12f86d30 service neutron] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:19.961747 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:26:20.000229 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3199c5fc-fb9c-48b7-bc0e-d1ba619349d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:20.001135 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3199c5fc-fb9c-48b7-bc0e-d1ba619349d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:20.001416 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3199c5fc-fb9c-48b7-bc0e-d1ba619349d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:20.001699 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3199c5fc-fb9c-48b7-bc0e-d1ba619349d8 admin admin] PATH_INFO: `/v3/users/ab8685ef23404ded9e4480f2b3ea3320` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:20.002901 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3199c5fc-fb9c-48b7-bc0e-d1ba619349d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab8685ef23404ded9e4480f2b3ea3320)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:20.004854 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3199c5fc-fb9c-48b7-bc0e-d1ba619349d8 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:20.195194 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-3199c5fc-fb9c-48b7-bc0e-d1ba619349d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab8685ef23404ded9e4480f2b3ea3320'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:20.259290 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-3199c5fc-fb9c-48b7-bc0e-d1ba619349d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab8685ef23404ded9e4480f2b3ea3320'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:20.260652 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-3199c5fc-fb9c-48b7-bc0e-d1ba619349d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab8685ef23404ded9e4480f2b3ea3320'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:20.275194 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-3199c5fc-fb9c-48b7-bc0e-d1ba619349d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab8685ef23404ded9e4480f2b3ea3320'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:20.275898 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-3199c5fc-fb9c-48b7-bc0e-d1ba619349d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab8685ef23404ded9e4480f2b3ea3320'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:20.288126 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 981/1969] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jun 29 11:26:19 2022] DELETE /identity/v3/users/ab8685ef23404ded9e4480f2b3ea3320 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:26:20.371572 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4479ac37-ee25-4026-baf9-9ac0be9ba39f admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:20.374953 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-4479ac37-ee25-4026-baf9-9ac0be9ba39f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:26:20.386394 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4479ac37-ee25-4026-baf9-9ac0be9ba39f admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:20.445371 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d50e4a7-e094-4d21-87c0-ff0821295d96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:20.446385 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d50e4a7-e094-4d21-87c0-ff0821295d96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:20.446565 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d50e4a7-e094-4d21-87c0-ff0821295d96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:20.446868 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d50e4a7-e094-4d21-87c0-ff0821295d96 admin admin] PATH_INFO: `/v3/users/8ae43b7540dd450e9a19eb0b2c85af8e` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:20.448678 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d50e4a7-e094-4d21-87c0-ff0821295d96 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ae43b7540dd450e9a19eb0b2c85af8e)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:20.450790 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d50e4a7-e094-4d21-87c0-ff0821295d96 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:20.559893 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:26:20.659480 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-2d50e4a7-e094-4d21-87c0-ff0821295d96 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ae43b7540dd450e9a19eb0b2c85af8e'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:20.660193 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-2d50e4a7-e094-4d21-87c0-ff0821295d96 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ae43b7540dd450e9a19eb0b2c85af8e'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:20.694802 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3199c5fc-fb9c-48b7-bc0e-d1ba619349d8 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:20.694802 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-3199c5fc-fb9c-48b7-bc0e-d1ba619349d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:26:20.723161 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3199c5fc-fb9c-48b7-bc0e-d1ba619349d8 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:20.723161 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35735084-b064-446f-8736-88b80691c318 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:20.723161 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35735084-b064-446f-8736-88b80691c318 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:20.723161 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35735084-b064-446f-8736-88b80691c318 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:20.723487 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35735084-b064-446f-8736-88b80691c318 admin admin] PATH_INFO: `/v3/projects/19943531ab424e9a93d8813017e9db58` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:20.723487 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35735084-b064-446f-8736-88b80691c318 admin admin] RBAC: Authorizing `identity:delete_project(project_id=19943531ab424e9a93d8813017e9db58)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:20.723487 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35735084-b064-446f-8736-88b80691c318 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:20.770715 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-2d50e4a7-e094-4d21-87c0-ff0821295d96 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ae43b7540dd450e9a19eb0b2c85af8e'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:20.794240 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-2d50e4a7-e094-4d21-87c0-ff0821295d96 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ae43b7540dd450e9a19eb0b2c85af8e'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:20.795220 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-2d50e4a7-e094-4d21-87c0-ff0821295d96 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ae43b7540dd450e9a19eb0b2c85af8e'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:20.814825 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 989/1970] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jun 29 11:26:20 2022] DELETE /identity/v3/users/8ae43b7540dd450e9a19eb0b2c85af8e => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:26:20.819514 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d36961-238f-41fc-9d33-985c9eeea521 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:20.819896 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d36961-238f-41fc-9d33-985c9eeea521 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:20.820543 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d36961-238f-41fc-9d33-985c9eeea521 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:20.838199 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-a3d36961-238f-41fc-9d33-985c9eeea521 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:26:20.839398 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-a3d36961-238f-41fc-9d33-985c9eeea521 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:26:20.875665 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-35735084-b064-446f-8736-88b80691c318 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '19943531ab424e9a93d8813017e9db58', 'request_id': 'req-35735084-b064-446f-8736-88b80691c318'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:20.902417 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 990/1971] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:26:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 11:26:20.917364 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d36961-238f-41fc-9d33-985c9eeea521 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:20.923238 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-a3d36961-238f-41fc-9d33-985c9eeea521 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:26:20.941529 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-35735084-b064-446f-8736-88b80691c318 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '19943531ab424e9a93d8813017e9db58', 'request_id': 'req-35735084-b064-446f-8736-88b80691c318'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:20.949142 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-35735084-b064-446f-8736-88b80691c318 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '19943531ab424e9a93d8813017e9db58', 'request_id': 'req-35735084-b064-446f-8736-88b80691c318'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:20.960648 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-35735084-b064-446f-8736-88b80691c318 admin admin] The token cache is being invalidate because project 19943531ab424e9a93d8813017e9db58 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 11:26:20.960939 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-35735084-b064-446f-8736-88b80691c318 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:20.961698 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-35735084-b064-446f-8736-88b80691c318 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:20.963817 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 982/1972] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jun 29 11:26:20 2022] DELETE /identity/v3/projects/19943531ab424e9a93d8813017e9db58 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 11:26:20.969979 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cef3397-f5c2-456e-addf-2d3709624481 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:20.970431 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cef3397-f5c2-456e-addf-2d3709624481 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:20.970721 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cef3397-f5c2-456e-addf-2d3709624481 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:20.982589 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d36961-238f-41fc-9d33-985c9eeea521 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:20.991111 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-8cef3397-f5c2-456e-addf-2d3709624481 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:26:20.992494 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-8cef3397-f5c2-456e-addf-2d3709624481 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:26:21.017106 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:26:21.040684 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e922c1c-02c7-4cbb-840a-eb6101081df9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f4bb9f989e4024a5da28dced646b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:21.041940 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e922c1c-02c7-4cbb-840a-eb6101081df9 service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:21.042232 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e922c1c-02c7-4cbb-840a-eb6101081df9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:21.042595 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e922c1c-02c7-4cbb-840a-eb6101081df9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:21.044573 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-1e922c1c-02c7-4cbb-840a-eb6101081df9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:26:21.058769 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 983/1973] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:26:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 11:26:21.061470 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:26:21.067257 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53ae861-9f92-436a-96a6-49f807fe9e58 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:21.067555 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53ae861-9f92-436a-96a6-49f807fe9e58 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:21.068083 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53ae861-9f92-436a-96a6-49f807fe9e58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:21.090009 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-a53ae861-9f92-436a-96a6-49f807fe9e58 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:26:21.091397 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-a53ae861-9f92-436a-96a6-49f807fe9e58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:26:21.099710 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 984/1974] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:26:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 11:26:21.107604 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53ae861-9f92-436a-96a6-49f807fe9e58 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:21.113000 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e922c1c-02c7-4cbb-840a-eb6101081df9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:21.118794 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e922c1c-02c7-4cbb-840a-eb6101081df9 service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:21.118912 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53ae861-9f92-436a-96a6-49f807fe9e58 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:21.122189 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e0aeb9-31c3-4c5c-a1b3-5fc240d897bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:21.122855 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e0aeb9-31c3-4c5c-a1b3-5fc240d897bf admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:21.123143 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e0aeb9-31c3-4c5c-a1b3-5fc240d897bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:21.123426 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e0aeb9-31c3-4c5c-a1b3-5fc240d897bf admin admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:21.124197 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e0aeb9-31c3-4c5c-a1b3-5fc240d897bf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:26:21.124510 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e0aeb9-31c3-4c5c-a1b3-5fc240d897bf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:21.124811 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 991/1975] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:26:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:26:21.128361 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e0aeb9-31c3-4c5c-a1b3-5fc240d897bf admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:21.142580 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 985/1976] 10.222.0.44 () {58 vars in 1152 bytes} [Wed Jun 29 11:26:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 11:26:21.147440 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e922c1c-02c7-4cbb-840a-eb6101081df9 service neutron] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:21.157540 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e922c1c-02c7-4cbb-840a-eb6101081df9 service neutron] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:21.162404 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c9d480-9e71-4af3-b8cf-f3fb6eb89aa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:21.163073 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c9d480-9e71-4af3-b8cf-f3fb6eb89aa8 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:21.163403 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c9d480-9e71-4af3-b8cf-f3fb6eb89aa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:21.164141 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c9d480-9e71-4af3-b8cf-f3fb6eb89aa8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:21.164958 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c9d480-9e71-4af3-b8cf-f3fb6eb89aa8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:21.168096 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c9d480-9e71-4af3-b8cf-f3fb6eb89aa8 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:21.261837 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 992/1977] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jun 29 11:26:21 2022] POST /identity/v3/projects => generated 383 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:26:21.266200 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e0aeb9-31c3-4c5c-a1b3-5fc240d897bf admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:21.276938 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e0aeb9-31c3-4c5c-a1b3-5fc240d897bf admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:21.339383 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4dd650-58c8-432d-a736-5664ed53cb5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:21.340104 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4dd650-58c8-432d-a736-5664ed53cb5b admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:21.340394 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4dd650-58c8-432d-a736-5664ed53cb5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:21.340671 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4dd650-58c8-432d-a736-5664ed53cb5b admin admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:21.341381 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4dd650-58c8-432d-a736-5664ed53cb5b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:21.343321 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4dd650-58c8-432d-a736-5664ed53cb5b admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:21.453953 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 986/1978] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jun 29 11:26:21 2022] POST /identity/v3/users => generated 292 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 11:26:21.456492 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c9d480-9e71-4af3-b8cf-f3fb6eb89aa8 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:21.467947 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c9d480-9e71-4af3-b8cf-f3fb6eb89aa8 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:21.473076 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8071572a-a136-4b6f-93a1-bdfffd10696f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:21.473745 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8071572a-a136-4b6f-93a1-bdfffd10696f admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:21.474078 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8071572a-a136-4b6f-93a1-bdfffd10696f admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:21.474358 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8071572a-a136-4b6f-93a1-bdfffd10696f admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:21.475109 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8071572a-a136-4b6f-93a1-bdfffd10696f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:21.476775 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8071572a-a136-4b6f-93a1-bdfffd10696f admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:21.487981 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 993/1979] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:26:21 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 11:26:21.491817 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4dd650-58c8-432d-a736-5664ed53cb5b admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:21.501894 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4dd650-58c8-432d-a736-5664ed53cb5b admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:21.506500 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987b4ce1-86a3-41a1-b3bf-1f9260f28721 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:21.507151 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987b4ce1-86a3-41a1-b3bf-1f9260f28721 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:21.507438 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987b4ce1-86a3-41a1-b3bf-1f9260f28721 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:21.507716 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987b4ce1-86a3-41a1-b3bf-1f9260f28721 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:21.508482 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987b4ce1-86a3-41a1-b3bf-1f9260f28721 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:21.510582 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987b4ce1-86a3-41a1-b3bf-1f9260f28721 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:21.522071 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 987/1980] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:26:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 11:26:21.526736 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8071572a-a136-4b6f-93a1-bdfffd10696f admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:21.536899 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8071572a-a136-4b6f-93a1-bdfffd10696f admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:21.541744 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81999dc9-5f8d-4bb0-b560-205431d4cdbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:21.542722 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81999dc9-5f8d-4bb0-b560-205431d4cdbc admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:21.543010 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81999dc9-5f8d-4bb0-b560-205431d4cdbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:21.543285 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81999dc9-5f8d-4bb0-b560-205431d4cdbc admin admin] PATH_INFO: `/v3/projects/df8613e4278a4fc288e0d5c22a665ea9/users/cc0aed87380d4dbfacbed7a960d9ea2b/roles/e1c88eebe911432cab6a07fbeb84653b` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:21.557782 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81999dc9-5f8d-4bb0-b560-205431d4cdbc admin admin] RBAC: Authorizing `identity:create_grant(project_id=df8613e4278a4fc288e0d5c22a665ea9, user_id=cc0aed87380d4dbfacbed7a960d9ea2b, role_id=e1c88eebe911432cab6a07fbeb84653b)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:21.560553 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81999dc9-5f8d-4bb0-b560-205431d4cdbc admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:21.592375 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987b4ce1-86a3-41a1-b3bf-1f9260f28721 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:21.594222 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-987b4ce1-86a3-41a1-b3bf-1f9260f28721 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:26:21.608895 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987b4ce1-86a3-41a1-b3bf-1f9260f28721 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:21.615854 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091115a6-cce7-4585-ae92-4e64268e630d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:21.616803 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091115a6-cce7-4585-ae92-4e64268e630d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:21.617113 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091115a6-cce7-4585-ae92-4e64268e630d admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:21.617386 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091115a6-cce7-4585-ae92-4e64268e630d admin admin] PATH_INFO: `/v3/projects/d6e15d1877044eeaae711e2cce5fc256` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:21.618621 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091115a6-cce7-4585-ae92-4e64268e630d admin admin] RBAC: Authorizing `identity:delete_project(project_id=d6e15d1877044eeaae711e2cce5fc256)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:21.621434 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091115a6-cce7-4585-ae92-4e64268e630d admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:21.653482 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 994/1981] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jun 29 11:26:21 2022] PUT /identity/v3/projects/df8613e4278a4fc288e0d5c22a665ea9/users/cc0aed87380d4dbfacbed7a960d9ea2b/roles/e1c88eebe911432cab6a07fbeb84653b => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 11:26:21.665285 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a54c57-9954-40dd-b086-e4bbf95d6be1 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:21.665576 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a54c57-9954-40dd-b086-e4bbf95d6be1 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:21.665918 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a54c57-9954-40dd-b086-e4bbf95d6be1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:21.695142 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-c8a54c57-9954-40dd-b086-e4bbf95d6be1 None None] MFA Rules not processed for user `cc0aed87380d4dbfacbed7a960d9ea2b`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:26:21.696425 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-c8a54c57-9954-40dd-b086-e4bbf95d6be1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:26:21.724977 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-091115a6-cce7-4585-ae92-4e64268e630d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd6e15d1877044eeaae711e2cce5fc256', 'request_id': 'req-091115a6-cce7-4585-ae92-4e64268e630d'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:21.767329 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 995/1982] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:26:21 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 11:26:21.778683 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a54c57-9954-40dd-b086-e4bbf95d6be1 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:21.780130 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-c8a54c57-9954-40dd-b086-e4bbf95d6be1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:26:21.796620 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-091115a6-cce7-4585-ae92-4e64268e630d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd6e15d1877044eeaae711e2cce5fc256', 'request_id': 'req-091115a6-cce7-4585-ae92-4e64268e630d'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:21.804781 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-091115a6-cce7-4585-ae92-4e64268e630d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd6e15d1877044eeaae711e2cce5fc256', 'request_id': 'req-091115a6-cce7-4585-ae92-4e64268e630d'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:21.818926 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-091115a6-cce7-4585-ae92-4e64268e630d admin admin] The token cache is being invalidate because project d6e15d1877044eeaae711e2cce5fc256 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 11:26:21.819256 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-091115a6-cce7-4585-ae92-4e64268e630d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:21.820392 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-091115a6-cce7-4585-ae92-4e64268e630d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:21.823483 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 988/1983] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jun 29 11:26:21 2022] DELETE /identity/v3/projects/d6e15d1877044eeaae711e2cce5fc256 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 11:26:21.827191 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091115a6-cce7-4585-ae92-4e64268e630d admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:21.831123 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-091115a6-cce7-4585-ae92-4e64268e630d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:26:21.841343 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:26:21.846523 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:26:21.862431 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a54c57-9954-40dd-b086-e4bbf95d6be1 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:21.864191 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314a76e7-e089-499c-bf6f-39c1629d6b5a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff7931b75a924394b1af733febce0834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:21.864849 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314a76e7-e089-499c-bf6f-39c1629d6b5a service cinder] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:21.865134 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314a76e7-e089-499c-bf6f-39c1629d6b5a service cinder] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:21.865411 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314a76e7-e089-499c-bf6f-39c1629d6b5a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:21.867295 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-314a76e7-e089-499c-bf6f-39c1629d6b5a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:26:21.887300 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314a76e7-e089-499c-bf6f-39c1629d6b5a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:21.888800 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314a76e7-e089-499c-bf6f-39c1629d6b5a service cinder] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:21.899103 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 996/1984] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:26:21 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 11:26:21.936981 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314a76e7-e089-499c-bf6f-39c1629d6b5a service cinder] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:21.936981 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-314a76e7-e089-499c-bf6f-39c1629d6b5a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:26:21.938525 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091115a6-cce7-4585-ae92-4e64268e630d admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:21.943420 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b069932-fa04-4992-a562-29ea9e371268 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:21.944320 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b069932-fa04-4992-a562-29ea9e371268 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:21.945199 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b069932-fa04-4992-a562-29ea9e371268 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:21.945199 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b069932-fa04-4992-a562-29ea9e371268 admin admin] PATH_INFO: `/v3/users/71569f5a94df46029d94b1cd33f6211b` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:21.946133 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b069932-fa04-4992-a562-29ea9e371268 admin admin] RBAC: Authorizing `identity:delete_user(user_id=71569f5a94df46029d94b1cd33f6211b)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:21.948691 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b069932-fa04-4992-a562-29ea9e371268 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:21.949970 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314a76e7-e089-499c-bf6f-39c1629d6b5a service cinder] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:21.955771 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cd97e9-cf05-44bc-a672-ce5fd85577d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:21.956758 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cd97e9-cf05-44bc-a672-ce5fd85577d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:21.957219 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cd97e9-cf05-44bc-a672-ce5fd85577d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:21.957568 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cd97e9-cf05-44bc-a672-ce5fd85577d9 admin admin] PATH_INFO: `/v3/users/cc0aed87380d4dbfacbed7a960d9ea2b` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:21.958947 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cd97e9-cf05-44bc-a672-ce5fd85577d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc0aed87380d4dbfacbed7a960d9ea2b)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:21.960996 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cd97e9-cf05-44bc-a672-ce5fd85577d9 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:22.134335 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-7b069932-fa04-4992-a562-29ea9e371268 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '71569f5a94df46029d94b1cd33f6211b'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:22.163506 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-a2cd97e9-cf05-44bc-a672-ce5fd85577d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc0aed87380d4dbfacbed7a960d9ea2b'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:22.164124 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-a2cd97e9-cf05-44bc-a672-ce5fd85577d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc0aed87380d4dbfacbed7a960d9ea2b'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:22.230587 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-7b069932-fa04-4992-a562-29ea9e371268 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '71569f5a94df46029d94b1cd33f6211b'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:22.231261 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-7b069932-fa04-4992-a562-29ea9e371268 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '71569f5a94df46029d94b1cd33f6211b'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:22.243721 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-7b069932-fa04-4992-a562-29ea9e371268 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '71569f5a94df46029d94b1cd33f6211b'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:22.244949 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-7b069932-fa04-4992-a562-29ea9e371268 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '71569f5a94df46029d94b1cd33f6211b'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:22.257103 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 989/1985] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jun 29 11:26:21 2022] DELETE /identity/v3/users/71569f5a94df46029d94b1cd33f6211b => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:26:22.275780 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-a2cd97e9-cf05-44bc-a672-ce5fd85577d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc0aed87380d4dbfacbed7a960d9ea2b'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:22.283595 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-a2cd97e9-cf05-44bc-a672-ce5fd85577d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc0aed87380d4dbfacbed7a960d9ea2b'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:22.284323 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-a2cd97e9-cf05-44bc-a672-ce5fd85577d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc0aed87380d4dbfacbed7a960d9ea2b'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:22.303396 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 997/1986] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jun 29 11:26:21 2022] DELETE /identity/v3/users/cc0aed87380d4dbfacbed7a960d9ea2b => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:26:22.308543 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca1d565-9378-4912-8e52-f00949daa29e None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:22.308839 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca1d565-9378-4912-8e52-f00949daa29e None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:22.309121 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca1d565-9378-4912-8e52-f00949daa29e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:22.332799 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-0ca1d565-9378-4912-8e52-f00949daa29e None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:26:22.334027 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-0ca1d565-9378-4912-8e52-f00949daa29e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:26:22.392669 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 990/1987] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:26:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:26:22.402773 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cd97e9-cf05-44bc-a672-ce5fd85577d9 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:22.404264 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-a2cd97e9-cf05-44bc-a672-ce5fd85577d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:26:22.461519 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cd97e9-cf05-44bc-a672-ce5fd85577d9 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:22.465838 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c107715b-547e-4b9f-b16a-89ee5c6d27b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f4bb9f989e4024a5da28dced646b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:22.466493 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c107715b-547e-4b9f-b16a-89ee5c6d27b3 service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:22.466777 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c107715b-547e-4b9f-b16a-89ee5c6d27b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:22.467082 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c107715b-547e-4b9f-b16a-89ee5c6d27b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:22.477115 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c107715b-547e-4b9f-b16a-89ee5c6d27b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:22.479189 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c107715b-547e-4b9f-b16a-89ee5c6d27b3 service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:22.489577 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 998/1988] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:26:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:26:22.680387 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca1d565-9378-4912-8e52-f00949daa29e None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:22.681857 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-0ca1d565-9378-4912-8e52-f00949daa29e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:26:22.699245 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca1d565-9378-4912-8e52-f00949daa29e None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:22.711612 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b928e2d3-714b-413d-bf4e-d19ca5c77fa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:22.714384 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b928e2d3-714b-413d-bf4e-d19ca5c77fa9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:22.714747 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b928e2d3-714b-413d-bf4e-d19ca5c77fa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:22.715147 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b928e2d3-714b-413d-bf4e-d19ca5c77fa9 admin admin] PATH_INFO: `/v3/projects/781ee553cbe04cc8b7f147d6bf399ebe` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:22.716628 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b928e2d3-714b-413d-bf4e-d19ca5c77fa9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=781ee553cbe04cc8b7f147d6bf399ebe)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:22.719676 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b928e2d3-714b-413d-bf4e-d19ca5c77fa9 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:22.747561 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:26:22.837031 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-b928e2d3-714b-413d-bf4e-d19ca5c77fa9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '781ee553cbe04cc8b7f147d6bf399ebe', 'request_id': 'req-b928e2d3-714b-413d-bf4e-d19ca5c77fa9'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:22.903939 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c107715b-547e-4b9f-b16a-89ee5c6d27b3 service neutron] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:22.905517 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-c107715b-547e-4b9f-b16a-89ee5c6d27b3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:26:22.919161 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c107715b-547e-4b9f-b16a-89ee5c6d27b3 service neutron] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:22.941839 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-b928e2d3-714b-413d-bf4e-d19ca5c77fa9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '781ee553cbe04cc8b7f147d6bf399ebe', 'request_id': 'req-b928e2d3-714b-413d-bf4e-d19ca5c77fa9'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:22.950545 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-b928e2d3-714b-413d-bf4e-d19ca5c77fa9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '781ee553cbe04cc8b7f147d6bf399ebe', 'request_id': 'req-b928e2d3-714b-413d-bf4e-d19ca5c77fa9'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:22.962186 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-b928e2d3-714b-413d-bf4e-d19ca5c77fa9 admin admin] The token cache is being invalidate because project 781ee553cbe04cc8b7f147d6bf399ebe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 11:26:22.962539 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-b928e2d3-714b-413d-bf4e-d19ca5c77fa9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:22.963929 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-b928e2d3-714b-413d-bf4e-d19ca5c77fa9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:22.965833 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 991/1989] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jun 29 11:26:22 2022] DELETE /identity/v3/projects/781ee553cbe04cc8b7f147d6bf399ebe => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:26:22.971938 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab13fb2-d572-47b3-b61b-94f7a894103d None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:22.972856 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab13fb2-d572-47b3-b61b-94f7a894103d None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:22.973149 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab13fb2-d572-47b3-b61b-94f7a894103d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:22.993966 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-0ab13fb2-d572-47b3-b61b-94f7a894103d None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:26:22.995180 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-0ab13fb2-d572-47b3-b61b-94f7a894103d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:26:22.996959 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f5e72f-c81a-49ad-8913-d80b6d8642e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:22.997837 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f5e72f-c81a-49ad-8913-d80b6d8642e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:22.998115 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f5e72f-c81a-49ad-8913-d80b6d8642e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:22.998392 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f5e72f-c81a-49ad-8913-d80b6d8642e0 admin admin] PATH_INFO: `/v3/projects/df8613e4278a4fc288e0d5c22a665ea9` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:22.999570 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f5e72f-c81a-49ad-8913-d80b6d8642e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=df8613e4278a4fc288e0d5c22a665ea9)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:23.001559 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f5e72f-c81a-49ad-8913-d80b6d8642e0 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:23.003059 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:26:23.003350 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 992/1990] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:26:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 11:26:23.013148 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68745699-3232-4f35-be13-039ce0d5bdaf None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:23.013148 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68745699-3232-4f35-be13-039ce0d5bdaf None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:23.013404 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68745699-3232-4f35-be13-039ce0d5bdaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:23.045464 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-68745699-3232-4f35-be13-039ce0d5bdaf None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:26:23.046734 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-68745699-3232-4f35-be13-039ce0d5bdaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:26:23.055749 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 993/1991] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:26:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 11:26:23.061444 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68745699-3232-4f35-be13-039ce0d5bdaf None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:23.071470 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68745699-3232-4f35-be13-039ce0d5bdaf None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:23.077529 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af37999-c8db-40c8-9941-4e40104ba676 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:23.079599 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af37999-c8db-40c8-9941-4e40104ba676 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:23.079599 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af37999-c8db-40c8-9941-4e40104ba676 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:23.079599 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af37999-c8db-40c8-9941-4e40104ba676 admin admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:23.079599 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af37999-c8db-40c8-9941-4e40104ba676 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:26:23.080240 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af37999-c8db-40c8-9941-4e40104ba676 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:23.081473 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af37999-c8db-40c8-9941-4e40104ba676 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:23.094910 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 994/1992] 10.222.0.44 () {58 vars in 1152 bytes} [Wed Jun 29 11:26:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 11:26:23.099223 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af37999-c8db-40c8-9941-4e40104ba676 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:23.109802 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af37999-c8db-40c8-9941-4e40104ba676 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:23.121642 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-45f5e72f-c81a-49ad-8913-d80b6d8642e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df8613e4278a4fc288e0d5c22a665ea9', 'request_id': 'req-45f5e72f-c81a-49ad-8913-d80b6d8642e0'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:23.129320 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-45f5e72f-c81a-49ad-8913-d80b6d8642e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df8613e4278a4fc288e0d5c22a665ea9', 'request_id': 'req-45f5e72f-c81a-49ad-8913-d80b6d8642e0'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:23.136794 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-45f5e72f-c81a-49ad-8913-d80b6d8642e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df8613e4278a4fc288e0d5c22a665ea9', 'request_id': 'req-45f5e72f-c81a-49ad-8913-d80b6d8642e0'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:23.171380 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa7e26f-b9e8-4d6a-b647-f93d165e32a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:23.172070 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa7e26f-b9e8-4d6a-b647-f93d165e32a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:23.172351 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa7e26f-b9e8-4d6a-b647-f93d165e32a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:23.172628 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa7e26f-b9e8-4d6a-b647-f93d165e32a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:23.173365 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa7e26f-b9e8-4d6a-b647-f93d165e32a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:23.175244 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa7e26f-b9e8-4d6a-b647-f93d165e32a3 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:23.226549 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-45f5e72f-c81a-49ad-8913-d80b6d8642e0 admin admin] The token cache is being invalidate because project df8613e4278a4fc288e0d5c22a665ea9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 11:26:23.226872 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-45f5e72f-c81a-49ad-8913-d80b6d8642e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:23.227500 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-45f5e72f-c81a-49ad-8913-d80b6d8642e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:23.229127 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 999/1993] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jun 29 11:26:22 2022] DELETE /identity/v3/projects/df8613e4278a4fc288e0d5c22a665ea9 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:26:23.233811 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfae0865-0312-46cc-aed8-2f7e31819670 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:23.234096 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfae0865-0312-46cc-aed8-2f7e31819670 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:23.234369 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfae0865-0312-46cc-aed8-2f7e31819670 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:23.252701 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-cfae0865-0312-46cc-aed8-2f7e31819670 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:26:23.253890 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-cfae0865-0312-46cc-aed8-2f7e31819670 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:26:23.262266 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1000/1994] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:26:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 11:26:23.270191 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87085a38-57eb-408a-a851-0e3114533cb0 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:23.270478 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87085a38-57eb-408a-a851-0e3114533cb0 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:23.270759 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87085a38-57eb-408a-a851-0e3114533cb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:23.278985 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 995/1995] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jun 29 11:26:23 2022] POST /identity/v3/projects => generated 361 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 11:26:23.283361 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa7e26f-b9e8-4d6a-b647-f93d165e32a3 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:23.284970 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-eaa7e26f-b9e8-4d6a-b647-f93d165e32a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:26:23.288628 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-87085a38-57eb-408a-a851-0e3114533cb0 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:26:23.289791 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-87085a38-57eb-408a-a851-0e3114533cb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:26:23.347966 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa7e26f-b9e8-4d6a-b647-f93d165e32a3 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:23.352064 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1001/1996] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:26:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 11:26:23.353245 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d8f11c-7ed7-4182-b644-7203e8bc16a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:23.353956 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d8f11c-7ed7-4182-b644-7203e8bc16a7 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:23.354240 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d8f11c-7ed7-4182-b644-7203e8bc16a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:23.354628 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d8f11c-7ed7-4182-b644-7203e8bc16a7 admin admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:23.357400 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d8f11c-7ed7-4182-b644-7203e8bc16a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:23.357400 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d8f11c-7ed7-4182-b644-7203e8bc16a7 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:23.358281 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87085a38-57eb-408a-a851-0e3114533cb0 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:23.380375 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87085a38-57eb-408a-a851-0e3114533cb0 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:23.384578 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8332a7e3-fc43-419c-b7bd-edc3edc195ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:23.386515 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8332a7e3-fc43-419c-b7bd-edc3edc195ae admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:23.386798 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8332a7e3-fc43-419c-b7bd-edc3edc195ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:23.387075 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8332a7e3-fc43-419c-b7bd-edc3edc195ae admin admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:23.387799 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8332a7e3-fc43-419c-b7bd-edc3edc195ae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:26:23.388138 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8332a7e3-fc43-419c-b7bd-edc3edc195ae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:23.389727 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8332a7e3-fc43-419c-b7bd-edc3edc195ae admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:23.401559 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1002/1997] 10.222.0.44 () {58 vars in 1152 bytes} [Wed Jun 29 11:26:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:26:23.405919 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8332a7e3-fc43-419c-b7bd-edc3edc195ae admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:23.415338 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8332a7e3-fc43-419c-b7bd-edc3edc195ae admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:23.420069 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c2105f-077b-45ca-b3a5-c24c03c158b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:23.420756 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c2105f-077b-45ca-b3a5-c24c03c158b8 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:23.421049 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c2105f-077b-45ca-b3a5-c24c03c158b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:23.421330 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c2105f-077b-45ca-b3a5-c24c03c158b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:23.422004 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c2105f-077b-45ca-b3a5-c24c03c158b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:23.423864 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c2105f-077b-45ca-b3a5-c24c03c158b8 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:23.436312 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:DELETE=7,keystone:SELECT=88,keystone:INSERT=5 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:26:23.442817 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 996/1998] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jun 29 11:26:23 2022] POST /identity/v3/users => generated 281 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 11:26:23.446928 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d8f11c-7ed7-4182-b644-7203e8bc16a7 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:23.457293 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d8f11c-7ed7-4182-b644-7203e8bc16a7 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:23.461956 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f962488-9854-463a-96a8-dc8570877856 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:23.462544 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f962488-9854-463a-96a8-dc8570877856 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:23.462832 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f962488-9854-463a-96a8-dc8570877856 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:23.463111 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f962488-9854-463a-96a8-dc8570877856 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:23.463861 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f962488-9854-463a-96a8-dc8570877856 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:23.465481 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f962488-9854-463a-96a8-dc8570877856 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:23.475502 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 997/1999] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:26:23 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 11:26:23.479635 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f962488-9854-463a-96a8-dc8570877856 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:23.489092 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f962488-9854-463a-96a8-dc8570877856 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:23.493666 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8dd21c-e758-4fca-8aed-759515b1b663 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:23.495338 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8dd21c-e758-4fca-8aed-759515b1b663 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:23.495338 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8dd21c-e758-4fca-8aed-759515b1b663 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:23.495515 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8dd21c-e758-4fca-8aed-759515b1b663 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:23.496274 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8dd21c-e758-4fca-8aed-759515b1b663 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:23.497825 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8dd21c-e758-4fca-8aed-759515b1b663 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:23.507960 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 998/2000] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:26:23 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 11:26:23.511704 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8dd21c-e758-4fca-8aed-759515b1b663 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:23.523182 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1003/2001] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jun 29 11:26:23 2022] POST /identity/v3/projects => generated 383 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 11:26:23.523675 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8dd21c-e758-4fca-8aed-759515b1b663 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:23.524159 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c2105f-077b-45ca-b3a5-c24c03c158b8 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:23.577579 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:26:23.583517 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74e235f-49ad-4565-9025-ce52432697eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:23.584444 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74e235f-49ad-4565-9025-ce52432697eb admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:23.584738 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74e235f-49ad-4565-9025-ce52432697eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:23.585019 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74e235f-49ad-4565-9025-ce52432697eb admin admin] PATH_INFO: `/v3/projects/85d74aff57b747dd9529ac0c1c97fcfa/users/d581da65c1154f63819c84f9b5f5bd06/roles/e1c88eebe911432cab6a07fbeb84653b` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:23.592152 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c2105f-077b-45ca-b3a5-c24c03c158b8 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:23.596999 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ce6cb8-f2ff-4621-b518-f61b5862a088 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:23.597659 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ce6cb8-f2ff-4621-b518-f61b5862a088 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:23.597942 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ce6cb8-f2ff-4621-b518-f61b5862a088 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:23.598216 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ce6cb8-f2ff-4621-b518-f61b5862a088 admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:23.598902 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ce6cb8-f2ff-4621-b518-f61b5862a088 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:23.600316 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74e235f-49ad-4565-9025-ce52432697eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=85d74aff57b747dd9529ac0c1c97fcfa, user_id=d581da65c1154f63819c84f9b5f5bd06, role_id=e1c88eebe911432cab6a07fbeb84653b)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:23.603094 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74e235f-49ad-4565-9025-ce52432697eb admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:23.603255 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ce6cb8-f2ff-4621-b518-f61b5862a088 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:23.681364 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 999/2002] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jun 29 11:26:23 2022] PUT /identity/v3/projects/85d74aff57b747dd9529ac0c1c97fcfa/users/d581da65c1154f63819c84f9b5f5bd06/roles/e1c88eebe911432cab6a07fbeb84653b => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 11:26:23.691650 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb9cee3-d64d-4fe2-ad15-95b5d5e29ffa None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:23.692008 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb9cee3-d64d-4fe2-ad15-95b5d5e29ffa None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:23.692290 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb9cee3-d64d-4fe2-ad15-95b5d5e29ffa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:23.705186 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1004/2003] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jun 29 11:26:23 2022] POST /identity/v3/users => generated 292 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 11:26:23.710243 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ce6cb8-f2ff-4621-b518-f61b5862a088 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:23.723065 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ce6cb8-f2ff-4621-b518-f61b5862a088 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:23.723987 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-cbb9cee3-d64d-4fe2-ad15-95b5d5e29ffa None None] MFA Rules not processed for user `d581da65c1154f63819c84f9b5f5bd06`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:26:23.725230 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-cbb9cee3-d64d-4fe2-ad15-95b5d5e29ffa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:26:23.781030 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29567bc-de84-486c-a8d9-a22764b4a6bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:23.781708 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29567bc-de84-486c-a8d9-a22764b4a6bb admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:23.781995 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29567bc-de84-486c-a8d9-a22764b4a6bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:23.782273 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29567bc-de84-486c-a8d9-a22764b4a6bb admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:23.782996 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29567bc-de84-486c-a8d9-a22764b4a6bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:23.784632 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29567bc-de84-486c-a8d9-a22764b4a6bb admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:23.791367 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1000/2004] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:26:23 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 11:26:23.798758 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1005/2005] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:26:23 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 11:26:23.801310 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb9cee3-d64d-4fe2-ad15-95b5d5e29ffa None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:23.811929 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb9cee3-d64d-4fe2-ad15-95b5d5e29ffa None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:23.816507 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bf7b51-cbfd-473c-b5cf-dc76a7e9916c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:23.817167 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bf7b51-cbfd-473c-b5cf-dc76a7e9916c admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:23.817445 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bf7b51-cbfd-473c-b5cf-dc76a7e9916c admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:23.817721 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bf7b51-cbfd-473c-b5cf-dc76a7e9916c admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:23.818504 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bf7b51-cbfd-473c-b5cf-dc76a7e9916c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:23.820137 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bf7b51-cbfd-473c-b5cf-dc76a7e9916c admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:23.822949 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29567bc-de84-486c-a8d9-a22764b4a6bb admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:23.824687 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-e29567bc-de84-486c-a8d9-a22764b4a6bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:26:23.831031 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1001/2006] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:26:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 11:26:23.834933 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bf7b51-cbfd-473c-b5cf-dc76a7e9916c admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:23.845429 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bf7b51-cbfd-473c-b5cf-dc76a7e9916c admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:23.850232 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaac2406-532b-4b09-b398-32caad009890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:23.851089 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaac2406-532b-4b09-b398-32caad009890 admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:23.851370 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaac2406-532b-4b09-b398-32caad009890 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:23.851644 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaac2406-532b-4b09-b398-32caad009890 admin admin] PATH_INFO: `/v3/projects/2c8b9fe1a01c4cd9a44f6e6250a1d0de/users/211415dd958e4a43b99e0e144061d205/roles/e1c88eebe911432cab6a07fbeb84653b` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:23.868201 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaac2406-532b-4b09-b398-32caad009890 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c8b9fe1a01c4cd9a44f6e6250a1d0de, user_id=211415dd958e4a43b99e0e144061d205, role_id=e1c88eebe911432cab6a07fbeb84653b)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:23.871340 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaac2406-532b-4b09-b398-32caad009890 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:23.888365 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29567bc-de84-486c-a8d9-a22764b4a6bb admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:23.892146 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7811ddbf-3710-4418-bb2d-82aef3c750ea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff7931b75a924394b1af733febce0834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:23.892805 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7811ddbf-3710-4418-bb2d-82aef3c750ea service cinder] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:23.893092 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7811ddbf-3710-4418-bb2d-82aef3c750ea service cinder] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:23.893376 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7811ddbf-3710-4418-bb2d-82aef3c750ea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:23.902294 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7811ddbf-3710-4418-bb2d-82aef3c750ea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:23.903709 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7811ddbf-3710-4418-bb2d-82aef3c750ea service cinder] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:23.915312 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1006/2007] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:26:23 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 11:26:23.940032 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7811ddbf-3710-4418-bb2d-82aef3c750ea service cinder] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:23.951347 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7811ddbf-3710-4418-bb2d-82aef3c750ea service cinder] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:23.955939 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68735461-9771-406c-aa28-b1d7856e178d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:23.956800 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68735461-9771-406c-aa28-b1d7856e178d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:23.957085 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68735461-9771-406c-aa28-b1d7856e178d admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:23.957362 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68735461-9771-406c-aa28-b1d7856e178d admin admin] PATH_INFO: `/v3/users/d581da65c1154f63819c84f9b5f5bd06` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:23.958579 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68735461-9771-406c-aa28-b1d7856e178d admin admin] RBAC: Authorizing `identity:delete_user(user_id=d581da65c1154f63819c84f9b5f5bd06)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:23.960528 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68735461-9771-406c-aa28-b1d7856e178d admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:23.969783 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1002/2008] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jun 29 11:26:23 2022] PUT /identity/v3/projects/2c8b9fe1a01c4cd9a44f6e6250a1d0de/users/211415dd958e4a43b99e0e144061d205/roles/e1c88eebe911432cab6a07fbeb84653b => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 11:26:23.980155 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5751835-692a-4703-b4ac-75450f71c30d None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:23.980448 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5751835-692a-4703-b4ac-75450f71c30d None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:23.980718 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5751835-692a-4703-b4ac-75450f71c30d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:24.007018 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:26:24.014606 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-f5751835-692a-4703-b4ac-75450f71c30d None None] MFA Rules not processed for user `211415dd958e4a43b99e0e144061d205`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:26:24.014606 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-f5751835-692a-4703-b4ac-75450f71c30d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:26:24.075681 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1003/2009] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:26:23 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 11:26:24.086723 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5751835-692a-4703-b4ac-75450f71c30d None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:24.126756 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-68735461-9771-406c-aa28-b1d7856e178d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd581da65c1154f63819c84f9b5f5bd06'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:24.127428 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-68735461-9771-406c-aa28-b1d7856e178d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd581da65c1154f63819c84f9b5f5bd06'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:24.146980 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5751835-692a-4703-b4ac-75450f71c30d None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:24.151029 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcf153e-8021-4f65-9ccd-b697c1bb1eab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff7931b75a924394b1af733febce0834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:24.151701 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcf153e-8021-4f65-9ccd-b697c1bb1eab service cinder] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:24.152023 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcf153e-8021-4f65-9ccd-b697c1bb1eab service cinder] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:24.152305 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcf153e-8021-4f65-9ccd-b697c1bb1eab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:24.154130 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-bfcf153e-8021-4f65-9ccd-b697c1bb1eab service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:26:24.165581 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcf153e-8021-4f65-9ccd-b697c1bb1eab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:24.167028 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcf153e-8021-4f65-9ccd-b697c1bb1eab service cinder] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:24.175788 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1004/2010] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:26:24 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 11:26:24.204658 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-68735461-9771-406c-aa28-b1d7856e178d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd581da65c1154f63819c84f9b5f5bd06'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:24.212886 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-68735461-9771-406c-aa28-b1d7856e178d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd581da65c1154f63819c84f9b5f5bd06'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:24.213689 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-68735461-9771-406c-aa28-b1d7856e178d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd581da65c1154f63819c84f9b5f5bd06'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:24.222248 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=87,keystone:INSERT=4,keystone:DELETE=9 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:26:24.237022 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1007/2011] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jun 29 11:26:23 2022] DELETE /identity/v3/users/d581da65c1154f63819c84f9b5f5bd06 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 11:26:24.238940 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b53e5bb-77ed-4f23-90d8-e26537ee7638 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:24.239235 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b53e5bb-77ed-4f23-90d8-e26537ee7638 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:24.239516 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b53e5bb-77ed-4f23-90d8-e26537ee7638 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:24.256958 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-6b53e5bb-77ed-4f23-90d8-e26537ee7638 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:26:24.258181 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-6b53e5bb-77ed-4f23-90d8-e26537ee7638 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:26:24.323096 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1005/2012] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:26:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:26:24.333659 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68735461-9771-406c-aa28-b1d7856e178d admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:24.335136 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-68735461-9771-406c-aa28-b1d7856e178d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:26:24.391995 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68735461-9771-406c-aa28-b1d7856e178d admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:24.395513 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8faeaa-2d6f-453b-a858-7959f58273f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f4bb9f989e4024a5da28dced646b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:24.396258 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8faeaa-2d6f-453b-a858-7959f58273f0 service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:24.396545 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8faeaa-2d6f-453b-a858-7959f58273f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:24.396816 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8faeaa-2d6f-453b-a858-7959f58273f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:24.405636 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8faeaa-2d6f-453b-a858-7959f58273f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:24.407039 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8faeaa-2d6f-453b-a858-7959f58273f0 service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:24.417019 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1008/2013] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:26:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 11:26:24.863541 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b53e5bb-77ed-4f23-90d8-e26537ee7638 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:24.864760 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-6b53e5bb-77ed-4f23-90d8-e26537ee7638 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:26:24.904894 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b53e5bb-77ed-4f23-90d8-e26537ee7638 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:24.909846 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcc4cc3-4910-4762-8f46-8ada104c1101 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:24.912433 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcc4cc3-4910-4762-8f46-8ada104c1101 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:24.912433 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcc4cc3-4910-4762-8f46-8ada104c1101 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:24.912433 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcc4cc3-4910-4762-8f46-8ada104c1101 admin admin] PATH_INFO: `/v3/projects/85d74aff57b747dd9529ac0c1c97fcfa` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:24.912689 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcc4cc3-4910-4762-8f46-8ada104c1101 admin admin] RBAC: Authorizing `identity:delete_project(project_id=85d74aff57b747dd9529ac0c1c97fcfa)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:24.914455 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcc4cc3-4910-4762-8f46-8ada104c1101 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:25.026662 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:26:25.039909 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-6dcc4cc3-4910-4762-8f46-8ada104c1101 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '85d74aff57b747dd9529ac0c1c97fcfa', 'request_id': 'req-6dcc4cc3-4910-4762-8f46-8ada104c1101'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:25.104714 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-6dcc4cc3-4910-4762-8f46-8ada104c1101 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '85d74aff57b747dd9529ac0c1c97fcfa', 'request_id': 'req-6dcc4cc3-4910-4762-8f46-8ada104c1101'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:25.111419 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-6dcc4cc3-4910-4762-8f46-8ada104c1101 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '85d74aff57b747dd9529ac0c1c97fcfa', 'request_id': 'req-6dcc4cc3-4910-4762-8f46-8ada104c1101'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:25.121378 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-6dcc4cc3-4910-4762-8f46-8ada104c1101 admin admin] The token cache is being invalidate because project 85d74aff57b747dd9529ac0c1c97fcfa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 11:26:25.121744 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-6dcc4cc3-4910-4762-8f46-8ada104c1101 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:25.122364 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-6dcc4cc3-4910-4762-8f46-8ada104c1101 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:25.123905 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1006/2014] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jun 29 11:26:24 2022] DELETE /identity/v3/projects/85d74aff57b747dd9529ac0c1c97fcfa => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:26:25.128775 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830bd018-6432-4d0c-8880-d9a7730d74fb None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:25.129308 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830bd018-6432-4d0c-8880-d9a7730d74fb None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:25.129308 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830bd018-6432-4d0c-8880-d9a7730d74fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:25.149114 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-830bd018-6432-4d0c-8880-d9a7730d74fb None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:26:25.151159 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-830bd018-6432-4d0c-8880-d9a7730d74fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:26:25.210218 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1009/2015] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:26:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:26:25.218659 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276a2478-fc53-49d8-9b64-b7a74b130531 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:25.218955 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276a2478-fc53-49d8-9b64-b7a74b130531 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:25.219294 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276a2478-fc53-49d8-9b64-b7a74b130531 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:25.237438 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-276a2478-fc53-49d8-9b64-b7a74b130531 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:26:25.238648 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-276a2478-fc53-49d8-9b64-b7a74b130531 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:26:25.246438 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1007/2016] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:26:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:26:25.250916 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830bd018-6432-4d0c-8880-d9a7730d74fb None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:25.261135 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830bd018-6432-4d0c-8880-d9a7730d74fb None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:25.265759 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4baaf07-6077-4988-9640-1f6a28a5f54c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:25.266384 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4baaf07-6077-4988-9640-1f6a28a5f54c admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:25.266668 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4baaf07-6077-4988-9640-1f6a28a5f54c admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:25.266954 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4baaf07-6077-4988-9640-1f6a28a5f54c admin admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:25.267669 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4baaf07-6077-4988-9640-1f6a28a5f54c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:26:25.268012 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4baaf07-6077-4988-9640-1f6a28a5f54c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:25.269592 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4baaf07-6077-4988-9640-1f6a28a5f54c admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:25.281712 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1010/2017] 10.222.0.44 () {58 vars in 1152 bytes} [Wed Jun 29 11:26:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:26:25.286207 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276a2478-fc53-49d8-9b64-b7a74b130531 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:25.295944 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276a2478-fc53-49d8-9b64-b7a74b130531 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:25.301369 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6369afb-67f7-49c0-b731-8a744eea77e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:25.302116 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6369afb-67f7-49c0-b731-8a744eea77e0 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:25.302398 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6369afb-67f7-49c0-b731-8a744eea77e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:25.302670 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6369afb-67f7-49c0-b731-8a744eea77e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:25.303347 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6369afb-67f7-49c0-b731-8a744eea77e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:25.305291 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6369afb-67f7-49c0-b731-8a744eea77e0 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:25.386281 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1008/2018] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jun 29 11:26:25 2022] POST /identity/v3/projects => generated 375 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:26:25.387950 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4baaf07-6077-4988-9640-1f6a28a5f54c admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:25.398126 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4baaf07-6077-4988-9640-1f6a28a5f54c admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:25.455257 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b82972-ce8e-47ad-868f-1564df4b6bea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:25.455985 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b82972-ce8e-47ad-868f-1564df4b6bea admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:25.456274 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b82972-ce8e-47ad-868f-1564df4b6bea admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:25.456555 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b82972-ce8e-47ad-868f-1564df4b6bea admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:25.457249 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b82972-ce8e-47ad-868f-1564df4b6bea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:25.459166 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b82972-ce8e-47ad-868f-1564df4b6bea admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:25.548144 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1011/2019] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jun 29 11:26:25 2022] POST /identity/v3/users => generated 288 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 11:26:25.552142 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6369afb-67f7-49c0-b731-8a744eea77e0 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:25.562224 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6369afb-67f7-49c0-b731-8a744eea77e0 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:25.567053 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd26483-69ab-43a7-b4f3-4973303a00ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:25.567709 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd26483-69ab-43a7-b4f3-4973303a00ea admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:25.568031 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd26483-69ab-43a7-b4f3-4973303a00ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:25.568305 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd26483-69ab-43a7-b4f3-4973303a00ea admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:25.569020 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd26483-69ab-43a7-b4f3-4973303a00ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:25.570610 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd26483-69ab-43a7-b4f3-4973303a00ea admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:25.580743 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1009/2020] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:26:25 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 11:26:25.584535 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b82972-ce8e-47ad-868f-1564df4b6bea admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:25.595416 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b82972-ce8e-47ad-868f-1564df4b6bea admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:25.600249 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654861f4-1b16-4071-9988-c0ea8d99db64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:25.600893 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654861f4-1b16-4071-9988-c0ea8d99db64 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:25.601175 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654861f4-1b16-4071-9988-c0ea8d99db64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:25.601454 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654861f4-1b16-4071-9988-c0ea8d99db64 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:25.602185 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654861f4-1b16-4071-9988-c0ea8d99db64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:25.604427 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654861f4-1b16-4071-9988-c0ea8d99db64 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:25.606424 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:26:25.615449 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1012/2021] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:26:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 11:26:25.619628 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd26483-69ab-43a7-b4f3-4973303a00ea admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:25.629642 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd26483-69ab-43a7-b4f3-4973303a00ea admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:25.634365 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dd5283-8090-486a-a7b7-a7c3f382bf91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:25.635238 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dd5283-8090-486a-a7b7-a7c3f382bf91 admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:25.635523 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dd5283-8090-486a-a7b7-a7c3f382bf91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:25.635798 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dd5283-8090-486a-a7b7-a7c3f382bf91 admin admin] PATH_INFO: `/v3/projects/568e57a173c54524b5b65543fe78bc11/users/9a6ed848470540d099ccd83ced26c436/roles/e1c88eebe911432cab6a07fbeb84653b` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:25.651603 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61dd5283-8090-486a-a7b7-a7c3f382bf91 admin admin] RBAC: Authorizing `identity:create_grant(project_id=568e57a173c54524b5b65543fe78bc11, user_id=9a6ed848470540d099ccd83ced26c436, role_id=e1c88eebe911432cab6a07fbeb84653b)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:25.654306 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61dd5283-8090-486a-a7b7-a7c3f382bf91 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:25.733269 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1010/2022] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jun 29 11:26:25 2022] PUT /identity/v3/projects/568e57a173c54524b5b65543fe78bc11/users/9a6ed848470540d099ccd83ced26c436/roles/e1c88eebe911432cab6a07fbeb84653b => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:26:25.744940 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b61b0e-6468-4d68-bc81-fc0d851ee9e9 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:25.745253 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b61b0e-6468-4d68-bc81-fc0d851ee9e9 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:25.745661 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b61b0e-6468-4d68-bc81-fc0d851ee9e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:25.773309 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-92b61b0e-6468-4d68-bc81-fc0d851ee9e9 None None] MFA Rules not processed for user `9a6ed848470540d099ccd83ced26c436`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:26:25.774556 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-92b61b0e-6468-4d68-bc81-fc0d851ee9e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:26:25.838815 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1013/2023] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:26:25 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:26:25.849594 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dd5283-8090-486a-a7b7-a7c3f382bf91 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:25.851297 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-61dd5283-8090-486a-a7b7-a7c3f382bf91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:26:25.906441 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dd5283-8090-486a-a7b7-a7c3f382bf91 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:25.910017 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0220b592-2753-454e-bac6-3f3b2dc60b7a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff7931b75a924394b1af733febce0834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:25.910666 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0220b592-2753-454e-bac6-3f3b2dc60b7a service cinder] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:25.910941 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0220b592-2753-454e-bac6-3f3b2dc60b7a service cinder] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:25.911211 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0220b592-2753-454e-bac6-3f3b2dc60b7a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:25.920378 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0220b592-2753-454e-bac6-3f3b2dc60b7a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:25.921802 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0220b592-2753-454e-bac6-3f3b2dc60b7a service cinder] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:25.930986 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1011/2024] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:26:25 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:26:29.298693 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b61b0e-6468-4d68-bc81-fc0d851ee9e9 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:29.300212 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-92b61b0e-6468-4d68-bc81-fc0d851ee9e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:26:29.368214 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b61b0e-6468-4d68-bc81-fc0d851ee9e9 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:29.374095 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5209be-1334-4496-98bd-08fbab01b620 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:29.375492 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5209be-1334-4496-98bd-08fbab01b620 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:29.375889 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5209be-1334-4496-98bd-08fbab01b620 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:29.376208 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5209be-1334-4496-98bd-08fbab01b620 admin admin] PATH_INFO: `/v3/users/defbd94f64cd4861ab8aafb851206a8f` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:29.377526 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5209be-1334-4496-98bd-08fbab01b620 admin admin] RBAC: Authorizing `identity:delete_user(user_id=defbd94f64cd4861ab8aafb851206a8f)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:29.379666 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5209be-1334-4496-98bd-08fbab01b620 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:29.646924 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-6f5209be-1334-4496-98bd-08fbab01b620 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'defbd94f64cd4861ab8aafb851206a8f'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:29.647696 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-6f5209be-1334-4496-98bd-08fbab01b620 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'defbd94f64cd4861ab8aafb851206a8f'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:29.750108 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-6f5209be-1334-4496-98bd-08fbab01b620 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'defbd94f64cd4861ab8aafb851206a8f'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:29.757939 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-6f5209be-1334-4496-98bd-08fbab01b620 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'defbd94f64cd4861ab8aafb851206a8f'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:29.758816 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-6f5209be-1334-4496-98bd-08fbab01b620 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'defbd94f64cd4861ab8aafb851206a8f'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:29.778121 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1014/2025] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jun 29 11:26:29 2022] DELETE /identity/v3/users/defbd94f64cd4861ab8aafb851206a8f => generated 0 bytes in 479 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:26:29.783197 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8f6e79-997b-40cb-9593-5a3cc6429f8a None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:29.783679 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8f6e79-997b-40cb-9593-5a3cc6429f8a None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:29.783766 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8f6e79-997b-40cb-9593-5a3cc6429f8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:29.800990 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-2a8f6e79-997b-40cb-9593-5a3cc6429f8a None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:26:29.802201 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-2a8f6e79-997b-40cb-9593-5a3cc6429f8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:26:29.863709 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1012/2026] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:26:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:26:29.873462 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5209be-1334-4496-98bd-08fbab01b620 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:29.874933 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-6f5209be-1334-4496-98bd-08fbab01b620 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:26:29.884733 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:26:29.935260 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5209be-1334-4496-98bd-08fbab01b620 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:29.938964 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99d86c1-eeb2-4ebb-8f46-54d56620f8d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f4bb9f989e4024a5da28dced646b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:29.939612 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99d86c1-eeb2-4ebb-8f46-54d56620f8d9 service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:29.939938 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99d86c1-eeb2-4ebb-8f46-54d56620f8d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:29.940498 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99d86c1-eeb2-4ebb-8f46-54d56620f8d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:29.950726 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99d86c1-eeb2-4ebb-8f46-54d56620f8d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:29.952221 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99d86c1-eeb2-4ebb-8f46-54d56620f8d9 service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:29.961609 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1015/2027] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:26:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:26:30.408912 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8f6e79-997b-40cb-9593-5a3cc6429f8a None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:30.410358 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-2a8f6e79-997b-40cb-9593-5a3cc6429f8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:26:30.422779 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8f6e79-997b-40cb-9593-5a3cc6429f8a None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:30.428541 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4020206-3ef5-4187-8a5c-5f91423d8873 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:30.431329 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4020206-3ef5-4187-8a5c-5f91423d8873 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:30.431622 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4020206-3ef5-4187-8a5c-5f91423d8873 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:30.431940 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4020206-3ef5-4187-8a5c-5f91423d8873 admin admin] PATH_INFO: `/v3/projects/e13ccc357c7c4b14b7f7a3ec6fdbfeb4` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:30.433910 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4020206-3ef5-4187-8a5c-5f91423d8873 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e13ccc357c7c4b14b7f7a3ec6fdbfeb4)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:30.435932 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4020206-3ef5-4187-8a5c-5f91423d8873 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:30.441123 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:26:30.556391 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-c4020206-3ef5-4187-8a5c-5f91423d8873 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e13ccc357c7c4b14b7f7a3ec6fdbfeb4', 'request_id': 'req-c4020206-3ef5-4187-8a5c-5f91423d8873'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:30.644096 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-c4020206-3ef5-4187-8a5c-5f91423d8873 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e13ccc357c7c4b14b7f7a3ec6fdbfeb4', 'request_id': 'req-c4020206-3ef5-4187-8a5c-5f91423d8873'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:30.650692 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-c4020206-3ef5-4187-8a5c-5f91423d8873 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e13ccc357c7c4b14b7f7a3ec6fdbfeb4', 'request_id': 'req-c4020206-3ef5-4187-8a5c-5f91423d8873'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:30.661424 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-c4020206-3ef5-4187-8a5c-5f91423d8873 admin admin] The token cache is being invalidate because project e13ccc357c7c4b14b7f7a3ec6fdbfeb4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 11:26:30.661714 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-c4020206-3ef5-4187-8a5c-5f91423d8873 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:30.662346 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-c4020206-3ef5-4187-8a5c-5f91423d8873 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:30.663903 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1013/2028] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jun 29 11:26:30 2022] DELETE /identity/v3/projects/e13ccc357c7c4b14b7f7a3ec6fdbfeb4 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:26:30.668412 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99d86c1-eeb2-4ebb-8f46-54d56620f8d9 service neutron] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:30.678038 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99d86c1-eeb2-4ebb-8f46-54d56620f8d9 service neutron] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:30.736819 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171acb1f-4a48-4ae8-8367-bd58c5cf2bac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:30.737685 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171acb1f-4a48-4ae8-8367-bd58c5cf2bac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:30.737961 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171acb1f-4a48-4ae8-8367-bd58c5cf2bac admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:30.738231 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171acb1f-4a48-4ae8-8367-bd58c5cf2bac admin admin] PATH_INFO: `/v3/users/4a8af2f7029e45a69b0fbcb986bbeafd` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:30.739437 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171acb1f-4a48-4ae8-8367-bd58c5cf2bac admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a8af2f7029e45a69b0fbcb986bbeafd)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:30.741342 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171acb1f-4a48-4ae8-8367-bd58c5cf2bac admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:30.972682 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-171acb1f-4a48-4ae8-8367-bd58c5cf2bac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a8af2f7029e45a69b0fbcb986bbeafd'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:30.973355 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-171acb1f-4a48-4ae8-8367-bd58c5cf2bac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a8af2f7029e45a69b0fbcb986bbeafd'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:31.057951 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-171acb1f-4a48-4ae8-8367-bd58c5cf2bac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a8af2f7029e45a69b0fbcb986bbeafd'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:31.065797 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-171acb1f-4a48-4ae8-8367-bd58c5cf2bac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a8af2f7029e45a69b0fbcb986bbeafd'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:31.066419 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-171acb1f-4a48-4ae8-8367-bd58c5cf2bac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a8af2f7029e45a69b0fbcb986bbeafd'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:31.083248 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1016/2029] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jun 29 11:26:30 2022] DELETE /identity/v3/users/4a8af2f7029e45a69b0fbcb986bbeafd => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:26:31.679579 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4020206-3ef5-4187-8a5c-5f91423d8873 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:31.681130 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-c4020206-3ef5-4187-8a5c-5f91423d8873 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:26:31.694131 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4020206-3ef5-4187-8a5c-5f91423d8873 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:31.751070 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8955576-66a9-4c7d-b7c8-1b603618a173 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:31.751953 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8955576-66a9-4c7d-b7c8-1b603618a173 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:31.752237 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8955576-66a9-4c7d-b7c8-1b603618a173 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:31.752512 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8955576-66a9-4c7d-b7c8-1b603618a173 admin admin] PATH_INFO: `/v3/projects/3430f0551c3c45ceaf2436a191d0fb45` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:31.753764 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8955576-66a9-4c7d-b7c8-1b603618a173 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3430f0551c3c45ceaf2436a191d0fb45)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:31.755685 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8955576-66a9-4c7d-b7c8-1b603618a173 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:31.860987 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-b8955576-66a9-4c7d-b7c8-1b603618a173 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3430f0551c3c45ceaf2436a191d0fb45', 'request_id': 'req-b8955576-66a9-4c7d-b7c8-1b603618a173'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:31.930281 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-b8955576-66a9-4c7d-b7c8-1b603618a173 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3430f0551c3c45ceaf2436a191d0fb45', 'request_id': 'req-b8955576-66a9-4c7d-b7c8-1b603618a173'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:31.937442 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-b8955576-66a9-4c7d-b7c8-1b603618a173 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3430f0551c3c45ceaf2436a191d0fb45', 'request_id': 'req-b8955576-66a9-4c7d-b7c8-1b603618a173'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:31.948573 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-b8955576-66a9-4c7d-b7c8-1b603618a173 admin admin] The token cache is being invalidate because project 3430f0551c3c45ceaf2436a191d0fb45 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 11:26:31.948867 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-b8955576-66a9-4c7d-b7c8-1b603618a173 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:31.949573 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-b8955576-66a9-4c7d-b7c8-1b603618a173 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:31.951745 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1014/2030] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jun 29 11:26:31 2022] DELETE /identity/v3/projects/3430f0551c3c45ceaf2436a191d0fb45 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:26:31.956859 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7c3b0b-4ff8-4f03-af7f-f7f885711d97 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:31.957352 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7c3b0b-4ff8-4f03-af7f-f7f885711d97 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:31.957461 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7c3b0b-4ff8-4f03-af7f-f7f885711d97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:31.976703 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-ab7c3b0b-4ff8-4f03-af7f-f7f885711d97 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:26:31.977899 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-ab7c3b0b-4ff8-4f03-af7f-f7f885711d97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:26:32.002103 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:26:32.038800 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1017/2031] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:26:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 11:26:32.046850 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9badf977-3ee0-44c3-a614-1d0753265262 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:32.047148 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9badf977-3ee0-44c3-a614-1d0753265262 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:32.047434 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9badf977-3ee0-44c3-a614-1d0753265262 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:32.066031 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-9badf977-3ee0-44c3-a614-1d0753265262 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:26:32.067229 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-9badf977-3ee0-44c3-a614-1d0753265262 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:26:32.075776 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1015/2032] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:26:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:26:32.079961 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7c3b0b-4ff8-4f03-af7f-f7f885711d97 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:32.089134 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7c3b0b-4ff8-4f03-af7f-f7f885711d97 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:32.093636 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3665970-8336-4bc2-9bd4-a13b4dbf44fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:32.094298 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3665970-8336-4bc2-9bd4-a13b4dbf44fe admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:32.094845 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3665970-8336-4bc2-9bd4-a13b4dbf44fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:32.095117 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3665970-8336-4bc2-9bd4-a13b4dbf44fe admin admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:32.095852 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3665970-8336-4bc2-9bd4-a13b4dbf44fe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:26:32.096167 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3665970-8336-4bc2-9bd4-a13b4dbf44fe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:32.097802 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3665970-8336-4bc2-9bd4-a13b4dbf44fe admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:32.109681 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1018/2033] 10.222.0.44 () {58 vars in 1152 bytes} [Wed Jun 29 11:26:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:26:32.113855 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9badf977-3ee0-44c3-a614-1d0753265262 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:32.123407 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9badf977-3ee0-44c3-a614-1d0753265262 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:32.128009 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997dd01c-ebde-463d-9803-e71f8abe19b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:32.128651 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997dd01c-ebde-463d-9803-e71f8abe19b7 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:32.128935 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997dd01c-ebde-463d-9803-e71f8abe19b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:32.129213 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997dd01c-ebde-463d-9803-e71f8abe19b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:32.129906 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997dd01c-ebde-463d-9803-e71f8abe19b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:32.131793 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997dd01c-ebde-463d-9803-e71f8abe19b7 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:32.145036 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:26:32.206517 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1016/2034] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jun 29 11:26:32 2022] POST /identity/v3/projects => generated 365 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:26:32.210737 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3665970-8336-4bc2-9bd4-a13b4dbf44fe admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:32.220893 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3665970-8336-4bc2-9bd4-a13b4dbf44fe admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:32.276422 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69602178-71bd-43a5-98fd-7cf178cf102c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:32.277087 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69602178-71bd-43a5-98fd-7cf178cf102c admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:32.277373 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69602178-71bd-43a5-98fd-7cf178cf102c admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:32.277657 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69602178-71bd-43a5-98fd-7cf178cf102c admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:32.278337 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69602178-71bd-43a5-98fd-7cf178cf102c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:32.280271 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69602178-71bd-43a5-98fd-7cf178cf102c admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:32.344238 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1019/2035] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jun 29 11:26:32 2022] POST /identity/v3/users => generated 283 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:26:32.348537 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997dd01c-ebde-463d-9803-e71f8abe19b7 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:32.358977 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997dd01c-ebde-463d-9803-e71f8abe19b7 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:32.363567 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e056c5-8e17-49ad-bd57-55f1c3cc9005 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:32.364326 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e056c5-8e17-49ad-bd57-55f1c3cc9005 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:32.364609 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e056c5-8e17-49ad-bd57-55f1c3cc9005 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:32.364883 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e056c5-8e17-49ad-bd57-55f1c3cc9005 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:32.365610 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e056c5-8e17-49ad-bd57-55f1c3cc9005 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:32.367199 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e056c5-8e17-49ad-bd57-55f1c3cc9005 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:32.377943 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1017/2036] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:26:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 11:26:32.381672 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69602178-71bd-43a5-98fd-7cf178cf102c admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:32.392092 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69602178-71bd-43a5-98fd-7cf178cf102c admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:32.396727 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97177773-ca3e-4764-bd54-66b75c36aa87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:32.397377 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97177773-ca3e-4764-bd54-66b75c36aa87 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:32.397662 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97177773-ca3e-4764-bd54-66b75c36aa87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:32.397945 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97177773-ca3e-4764-bd54-66b75c36aa87 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:32.398670 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97177773-ca3e-4764-bd54-66b75c36aa87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:32.400295 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97177773-ca3e-4764-bd54-66b75c36aa87 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:32.410104 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1020/2037] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:26:32 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 11:26:32.414086 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e056c5-8e17-49ad-bd57-55f1c3cc9005 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:32.424271 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e056c5-8e17-49ad-bd57-55f1c3cc9005 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:32.428735 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900a81be-bc5c-4ec0-ba73-a1110948e9c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:32.429599 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900a81be-bc5c-4ec0-ba73-a1110948e9c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:32.429876 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900a81be-bc5c-4ec0-ba73-a1110948e9c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:32.430148 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900a81be-bc5c-4ec0-ba73-a1110948e9c4 admin admin] PATH_INFO: `/v3/projects/a1c3a4798c2d4250abb6b87812a630e5/users/c4ab39dfbe6d421b9aa35767d4f0d795/roles/e1c88eebe911432cab6a07fbeb84653b` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:32.444772 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900a81be-bc5c-4ec0-ba73-a1110948e9c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1c3a4798c2d4250abb6b87812a630e5, user_id=c4ab39dfbe6d421b9aa35767d4f0d795, role_id=e1c88eebe911432cab6a07fbeb84653b)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:32.447385 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900a81be-bc5c-4ec0-ba73-a1110948e9c4 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:32.565812 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1018/2038] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jun 29 11:26:32 2022] PUT /identity/v3/projects/a1c3a4798c2d4250abb6b87812a630e5/users/c4ab39dfbe6d421b9aa35767d4f0d795/roles/e1c88eebe911432cab6a07fbeb84653b => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:26:32.576469 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96622477-2474-4969-9453-f7e7d6762c9e None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:32.576763 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96622477-2474-4969-9453-f7e7d6762c9e None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:32.577042 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96622477-2474-4969-9453-f7e7d6762c9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:32.629383 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-96622477-2474-4969-9453-f7e7d6762c9e None None] MFA Rules not processed for user `c4ab39dfbe6d421b9aa35767d4f0d795`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:26:32.630605 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-96622477-2474-4969-9453-f7e7d6762c9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:26:32.701477 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1021/2039] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:26:32 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:26:32.706038 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900a81be-bc5c-4ec0-ba73-a1110948e9c4 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:32.772394 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900a81be-bc5c-4ec0-ba73-a1110948e9c4 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:32.777344 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913b0858-2182-409f-9f7a-b101e51689be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:32.778575 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913b0858-2182-409f-9f7a-b101e51689be admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:32.778859 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913b0858-2182-409f-9f7a-b101e51689be admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:32.779132 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913b0858-2182-409f-9f7a-b101e51689be admin admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:32.779850 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913b0858-2182-409f-9f7a-b101e51689be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:32.781769 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913b0858-2182-409f-9f7a-b101e51689be admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:32.885465 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1019/2040] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jun 29 11:26:32 2022] POST /identity/v3/projects => generated 365 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:26:32.889440 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96622477-2474-4969-9453-f7e7d6762c9e None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:32.900073 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96622477-2474-4969-9453-f7e7d6762c9e None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:32.922104 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:26:32.958681 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd914f2d-819c-4bbc-bf94-191425d6c17a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:32.959352 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd914f2d-819c-4bbc-bf94-191425d6c17a admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:32.959638 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd914f2d-819c-4bbc-bf94-191425d6c17a admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:32.959941 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd914f2d-819c-4bbc-bf94-191425d6c17a admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:32.960635 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd914f2d-819c-4bbc-bf94-191425d6c17a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:32.962529 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd914f2d-819c-4bbc-bf94-191425d6c17a admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:33.066631 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1022/2041] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jun 29 11:26:32 2022] POST /identity/v3/users => generated 289 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:26:33.070842 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913b0858-2182-409f-9f7a-b101e51689be admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:33.081289 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913b0858-2182-409f-9f7a-b101e51689be admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:33.086020 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079b94a1-2a9d-4b40-b15d-7db4819bf2e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:33.086675 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079b94a1-2a9d-4b40-b15d-7db4819bf2e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:33.086960 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079b94a1-2a9d-4b40-b15d-7db4819bf2e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:33.087236 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079b94a1-2a9d-4b40-b15d-7db4819bf2e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:33.088014 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079b94a1-2a9d-4b40-b15d-7db4819bf2e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:33.089657 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079b94a1-2a9d-4b40-b15d-7db4819bf2e2 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:33.100776 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1020/2042] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:26:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 11:26:33.104799 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd914f2d-819c-4bbc-bf94-191425d6c17a admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:33.115287 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd914f2d-819c-4bbc-bf94-191425d6c17a admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:33.119773 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e30d01-8e51-428c-aed9-d59b088fb2f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:33.120672 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e30d01-8e51-428c-aed9-d59b088fb2f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:33.120957 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e30d01-8e51-428c-aed9-d59b088fb2f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:33.121236 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e30d01-8e51-428c-aed9-d59b088fb2f3 admin admin] PATH_INFO: `/v3/projects/2ecbd73a067b46249fb587ced926d25a/users/331c576baffc429baec7d5058c8fdd1a/roles/e95c2ce63e114cc19a9dda6ad21e794f` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:33.135708 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e30d01-8e51-428c-aed9-d59b088fb2f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ecbd73a067b46249fb587ced926d25a, user_id=331c576baffc429baec7d5058c8fdd1a, role_id=e95c2ce63e114cc19a9dda6ad21e794f)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:33.138363 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e30d01-8e51-428c-aed9-d59b088fb2f3 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:33.214941 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1023/2043] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jun 29 11:26:33 2022] PUT /identity/v3/projects/2ecbd73a067b46249fb587ced926d25a/users/331c576baffc429baec7d5058c8fdd1a/roles/e95c2ce63e114cc19a9dda6ad21e794f => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 11:26:33.219387 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079b94a1-2a9d-4b40-b15d-7db4819bf2e2 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:33.229567 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079b94a1-2a9d-4b40-b15d-7db4819bf2e2 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:33.284447 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d27079-11f9-4b9f-8463-a5deb8f7c620 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:33.285095 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d27079-11f9-4b9f-8463-a5deb8f7c620 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:33.285375 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d27079-11f9-4b9f-8463-a5deb8f7c620 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:33.285652 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d27079-11f9-4b9f-8463-a5deb8f7c620 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:33.286464 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d27079-11f9-4b9f-8463-a5deb8f7c620 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:33.288072 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d27079-11f9-4b9f-8463-a5deb8f7c620 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:33.297431 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1021/2044] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:26:33 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 11:26:33.301528 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e30d01-8e51-428c-aed9-d59b088fb2f3 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:33.311664 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e30d01-8e51-428c-aed9-d59b088fb2f3 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:33.316240 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc71c11-1646-4426-abf0-3d06f1475b05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:33.317119 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc71c11-1646-4426-abf0-3d06f1475b05 admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:33.317406 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc71c11-1646-4426-abf0-3d06f1475b05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:33.317684 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc71c11-1646-4426-abf0-3d06f1475b05 admin admin] PATH_INFO: `/v3/projects/2ecbd73a067b46249fb587ced926d25a/users/331c576baffc429baec7d5058c8fdd1a/roles/e95c2ce63e114cc19a9dda6ad21e794f` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:33.319649 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc71c11-1646-4426-abf0-3d06f1475b05 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ecbd73a067b46249fb587ced926d25a, user_id=331c576baffc429baec7d5058c8fdd1a, role_id=e95c2ce63e114cc19a9dda6ad21e794f)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:33.322309 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc71c11-1646-4426-abf0-3d06f1475b05 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:33.332812 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1024/2045] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jun 29 11:26:33 2022] PUT /identity/v3/projects/2ecbd73a067b46249fb587ced926d25a/users/331c576baffc429baec7d5058c8fdd1a/roles/e95c2ce63e114cc19a9dda6ad21e794f => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 11:26:33.344258 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e495c7c4-8016-4909-82d9-396a57e115f7 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:33.344734 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e495c7c4-8016-4909-82d9-396a57e115f7 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:33.345021 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e495c7c4-8016-4909-82d9-396a57e115f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:33.371258 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-e495c7c4-8016-4909-82d9-396a57e115f7 None None] MFA Rules not processed for user `331c576baffc429baec7d5058c8fdd1a`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:26:33.372472 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-e495c7c4-8016-4909-82d9-396a57e115f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:26:33.389211 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:26:33.438389 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1022/2046] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:26:33 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:26:33.449749 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc71c11-1646-4426-abf0-3d06f1475b05 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:33.452097 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-6fc71c11-1646-4426-abf0-3d06f1475b05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:26:33.514046 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc71c11-1646-4426-abf0-3d06f1475b05 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:33.517906 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5434336c-c698-40e1-b617-91e5a64cdb2b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff7931b75a924394b1af733febce0834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:33.518549 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5434336c-c698-40e1-b617-91e5a64cdb2b service cinder] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:33.518828 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5434336c-c698-40e1-b617-91e5a64cdb2b service cinder] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:33.519106 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5434336c-c698-40e1-b617-91e5a64cdb2b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:33.528690 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5434336c-c698-40e1-b617-91e5a64cdb2b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:33.530128 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5434336c-c698-40e1-b617-91e5a64cdb2b service cinder] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:33.539306 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1025/2047] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:26:33 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:26:33.795482 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e495c7c4-8016-4909-82d9-396a57e115f7 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:33.805290 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e495c7c4-8016-4909-82d9-396a57e115f7 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:33.808976 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e47b0f1-0f37-4090-90b9-fb17983f0560 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff7931b75a924394b1af733febce0834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:33.810134 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e47b0f1-0f37-4090-90b9-fb17983f0560 service cinder] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:33.810419 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e47b0f1-0f37-4090-90b9-fb17983f0560 service cinder] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:33.810697 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e47b0f1-0f37-4090-90b9-fb17983f0560 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:33.889126 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e47b0f1-0f37-4090-90b9-fb17983f0560 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:33.890826 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e47b0f1-0f37-4090-90b9-fb17983f0560 service cinder] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:33.902490 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1023/2048] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:26:33 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:26:37.017696 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5434336c-c698-40e1-b617-91e5a64cdb2b service cinder] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:37.019110 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-5434336c-c698-40e1-b617-91e5a64cdb2b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:26:37.083951 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5434336c-c698-40e1-b617-91e5a64cdb2b service cinder] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:37.088990 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa96e411-33f6-4ae7-9591-7c54df8f105d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:37.089891 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa96e411-33f6-4ae7-9591-7c54df8f105d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:37.090202 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa96e411-33f6-4ae7-9591-7c54df8f105d admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:37.090476 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa96e411-33f6-4ae7-9591-7c54df8f105d admin admin] PATH_INFO: `/v3/users/9a6ed848470540d099ccd83ced26c436` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:37.091784 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa96e411-33f6-4ae7-9591-7c54df8f105d admin admin] RBAC: Authorizing `identity:delete_user(user_id=9a6ed848470540d099ccd83ced26c436)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:37.093976 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa96e411-33f6-4ae7-9591-7c54df8f105d admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:37.118353 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:26:37.308430 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-fa96e411-33f6-4ae7-9591-7c54df8f105d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9a6ed848470540d099ccd83ced26c436'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:37.309194 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-fa96e411-33f6-4ae7-9591-7c54df8f105d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9a6ed848470540d099ccd83ced26c436'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:37.432799 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-fa96e411-33f6-4ae7-9591-7c54df8f105d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9a6ed848470540d099ccd83ced26c436'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:37.449117 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-fa96e411-33f6-4ae7-9591-7c54df8f105d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9a6ed848470540d099ccd83ced26c436'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:37.449923 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-fa96e411-33f6-4ae7-9591-7c54df8f105d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9a6ed848470540d099ccd83ced26c436'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:37.487016 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1026/2049] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jun 29 11:26:37 2022] DELETE /identity/v3/users/9a6ed848470540d099ccd83ced26c436 => generated 0 bytes in 467 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:26:37.488733 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fe007a-7646-4fb5-aab3-8098d9f595e9 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:37.489191 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fe007a-7646-4fb5-aab3-8098d9f595e9 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:37.489352 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fe007a-7646-4fb5-aab3-8098d9f595e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:37.507168 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-67fe007a-7646-4fb5-aab3-8098d9f595e9 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:26:37.508492 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-67fe007a-7646-4fb5-aab3-8098d9f595e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:26:37.575718 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1024/2050] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:26:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 11:26:37.583223 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa96e411-33f6-4ae7-9591-7c54df8f105d admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:37.584891 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-fa96e411-33f6-4ae7-9591-7c54df8f105d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:26:37.647687 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa96e411-33f6-4ae7-9591-7c54df8f105d admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:37.651458 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c0da76-9b06-416c-8f4d-67cd22e4078f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f4bb9f989e4024a5da28dced646b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:37.652140 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c0da76-9b06-416c-8f4d-67cd22e4078f service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:37.652542 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c0da76-9b06-416c-8f4d-67cd22e4078f service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:37.652701 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c0da76-9b06-416c-8f4d-67cd22e4078f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:37.663966 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c0da76-9b06-416c-8f4d-67cd22e4078f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:37.664104 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c0da76-9b06-416c-8f4d-67cd22e4078f service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:37.674720 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1027/2051] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:26:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:26:38.143877 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fe007a-7646-4fb5-aab3-8098d9f595e9 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:38.145672 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-67fe007a-7646-4fb5-aab3-8098d9f595e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:26:38.158769 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fe007a-7646-4fb5-aab3-8098d9f595e9 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:38.163439 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ed25a6-7215-40d5-99a9-b0d51280c3bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:38.165314 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ed25a6-7215-40d5-99a9-b0d51280c3bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:38.165314 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ed25a6-7215-40d5-99a9-b0d51280c3bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:38.165314 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ed25a6-7215-40d5-99a9-b0d51280c3bc admin admin] PATH_INFO: `/v3/projects/568e57a173c54524b5b65543fe78bc11` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:38.166187 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ed25a6-7215-40d5-99a9-b0d51280c3bc admin admin] RBAC: Authorizing `identity:delete_project(project_id=568e57a173c54524b5b65543fe78bc11)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:38.168180 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ed25a6-7215-40d5-99a9-b0d51280c3bc admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:38.299168 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:26:38.300628 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-38ed25a6-7215-40d5-99a9-b0d51280c3bc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '568e57a173c54524b5b65543fe78bc11', 'request_id': 'req-38ed25a6-7215-40d5-99a9-b0d51280c3bc'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:38.406394 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-38ed25a6-7215-40d5-99a9-b0d51280c3bc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '568e57a173c54524b5b65543fe78bc11', 'request_id': 'req-38ed25a6-7215-40d5-99a9-b0d51280c3bc'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:38.415218 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-38ed25a6-7215-40d5-99a9-b0d51280c3bc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '568e57a173c54524b5b65543fe78bc11', 'request_id': 'req-38ed25a6-7215-40d5-99a9-b0d51280c3bc'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:38.426657 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-38ed25a6-7215-40d5-99a9-b0d51280c3bc admin admin] The token cache is being invalidate because project 568e57a173c54524b5b65543fe78bc11 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 11:26:38.426953 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-38ed25a6-7215-40d5-99a9-b0d51280c3bc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:38.428033 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-38ed25a6-7215-40d5-99a9-b0d51280c3bc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:38.433185 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1025/2052] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jun 29 11:26:38 2022] DELETE /identity/v3/projects/568e57a173c54524b5b65543fe78bc11 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:26:38.435781 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8087ef7-739a-4772-b674-a50edf2e8788 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:38.436249 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8087ef7-739a-4772-b674-a50edf2e8788 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:38.436358 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8087ef7-739a-4772-b674-a50edf2e8788 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:38.495632 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-c8087ef7-739a-4772-b674-a50edf2e8788 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:26:38.497009 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-c8087ef7-739a-4772-b674-a50edf2e8788 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:26:38.587034 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1028/2053] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:26:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:26:38.591957 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51320dc6-a83a-4cdb-861f-6d97b80b8c22 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:38.592857 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51320dc6-a83a-4cdb-861f-6d97b80b8c22 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:38.593218 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51320dc6-a83a-4cdb-861f-6d97b80b8c22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:38.620728 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-51320dc6-a83a-4cdb-861f-6d97b80b8c22 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:26:38.622803 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-51320dc6-a83a-4cdb-861f-6d97b80b8c22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:26:38.637855 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1026/2054] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:26:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 11:26:38.640310 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8087ef7-739a-4772-b674-a50edf2e8788 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:38.650391 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8087ef7-739a-4772-b674-a50edf2e8788 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:38.655527 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cef3a4a-a087-493e-a4a5-baed361e7313 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:38.656291 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cef3a4a-a087-493e-a4a5-baed361e7313 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:38.656576 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cef3a4a-a087-493e-a4a5-baed361e7313 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:38.664124 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cef3a4a-a087-493e-a4a5-baed361e7313 admin admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:38.665063 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cef3a4a-a087-493e-a4a5-baed361e7313 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:26:38.668805 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cef3a4a-a087-493e-a4a5-baed361e7313 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:38.668805 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cef3a4a-a087-493e-a4a5-baed361e7313 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:38.707141 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1029/2055] 10.222.0.44 () {58 vars in 1152 bytes} [Wed Jun 29 11:26:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:26:38.711644 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51320dc6-a83a-4cdb-861f-6d97b80b8c22 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:38.722158 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51320dc6-a83a-4cdb-861f-6d97b80b8c22 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:38.728192 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee5b305-ac90-4d43-b25b-90b621c4bf04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:38.729013 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee5b305-ac90-4d43-b25b-90b621c4bf04 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:38.729459 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee5b305-ac90-4d43-b25b-90b621c4bf04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:38.729940 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee5b305-ac90-4d43-b25b-90b621c4bf04 admin admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:38.730692 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee5b305-ac90-4d43-b25b-90b621c4bf04 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:38.733492 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee5b305-ac90-4d43-b25b-90b621c4bf04 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:38.843871 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1027/2056] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jun 29 11:26:38 2022] POST /identity/v3/projects => generated 371 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:26:38.846403 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cef3a4a-a087-493e-a4a5-baed361e7313 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:38.858111 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cef3a4a-a087-493e-a4a5-baed361e7313 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:38.904150 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:26:38.920402 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee5b305-ac90-4d43-b25b-90b621c4bf04 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:38.923493 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-4ee5b305-ac90-4d43-b25b-90b621c4bf04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:26:38.973726 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ba2920-4f1e-481a-ad26-99198fde7c7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:38.974670 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ba2920-4f1e-481a-ad26-99198fde7c7e admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:38.974955 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ba2920-4f1e-481a-ad26-99198fde7c7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:38.975228 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ba2920-4f1e-481a-ad26-99198fde7c7e admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:38.976039 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ba2920-4f1e-481a-ad26-99198fde7c7e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:38.977973 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ba2920-4f1e-481a-ad26-99198fde7c7e admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:39.067167 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee5b305-ac90-4d43-b25b-90b621c4bf04 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:39.086508 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1030/2057] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jun 29 11:26:38 2022] POST /identity/v3/users => generated 286 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:26:39.086508 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ba2920-4f1e-481a-ad26-99198fde7c7e admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:39.090339 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c89f1a-121a-4805-906c-6e4daf74862b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:39.092190 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-03ba2920-4f1e-481a-ad26-99198fde7c7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:26:39.093521 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c89f1a-121a-4805-906c-6e4daf74862b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:39.095102 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c89f1a-121a-4805-906c-6e4daf74862b admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:39.095720 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c89f1a-121a-4805-906c-6e4daf74862b admin admin] PATH_INFO: `/v3/users/86ecc69eb95b4533b29d710b5c6c804c` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:39.100871 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c89f1a-121a-4805-906c-6e4daf74862b admin admin] RBAC: Authorizing `identity:delete_user(user_id=86ecc69eb95b4533b29d710b5c6c804c)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:39.103811 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c89f1a-121a-4805-906c-6e4daf74862b admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:39.120864 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ba2920-4f1e-481a-ad26-99198fde7c7e admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:39.130123 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3bab92-1be2-4bb6-9252-e1538e4fa612 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:39.131212 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3bab92-1be2-4bb6-9252-e1538e4fa612 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:39.131647 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3bab92-1be2-4bb6-9252-e1538e4fa612 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:39.132785 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3bab92-1be2-4bb6-9252-e1538e4fa612 admin admin] PATH_INFO: `/v3/users/211415dd958e4a43b99e0e144061d205` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:39.134695 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3bab92-1be2-4bb6-9252-e1538e4fa612 admin admin] RBAC: Authorizing `identity:delete_user(user_id=211415dd958e4a43b99e0e144061d205)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:39.138201 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3bab92-1be2-4bb6-9252-e1538e4fa612 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:39.393494 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-4d3bab92-1be2-4bb6-9252-e1538e4fa612 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '211415dd958e4a43b99e0e144061d205'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:39.394368 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-4d3bab92-1be2-4bb6-9252-e1538e4fa612 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '211415dd958e4a43b99e0e144061d205'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:39.415910 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-c8c89f1a-121a-4805-906c-6e4daf74862b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '86ecc69eb95b4533b29d710b5c6c804c'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:39.463896 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-4d3bab92-1be2-4bb6-9252-e1538e4fa612 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '211415dd958e4a43b99e0e144061d205'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:39.473684 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-4d3bab92-1be2-4bb6-9252-e1538e4fa612 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '211415dd958e4a43b99e0e144061d205'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:39.475018 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-4d3bab92-1be2-4bb6-9252-e1538e4fa612 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '211415dd958e4a43b99e0e144061d205'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:39.498256 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1031/2058] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jun 29 11:26:39 2022] DELETE /identity/v3/users/211415dd958e4a43b99e0e144061d205 => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:26:39.501463 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3bab92-1be2-4bb6-9252-e1538e4fa612 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:39.502934 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-4d3bab92-1be2-4bb6-9252-e1538e4fa612 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:26:39.566734 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3bab92-1be2-4bb6-9252-e1538e4fa612 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:39.571393 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39f1d91-e97b-42e1-9881-1a4927068386 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:39.572095 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39f1d91-e97b-42e1-9881-1a4927068386 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:39.572375 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39f1d91-e97b-42e1-9881-1a4927068386 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:39.572650 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39f1d91-e97b-42e1-9881-1a4927068386 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:39.573361 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39f1d91-e97b-42e1-9881-1a4927068386 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:39.574983 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39f1d91-e97b-42e1-9881-1a4927068386 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:39.584548 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1032/2059] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:26:39 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 11:26:39.589097 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-c8c89f1a-121a-4805-906c-6e4daf74862b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '86ecc69eb95b4533b29d710b5c6c804c'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:39.591754 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b091ef9-ed86-41dc-a6ec-651eb67dcbe9 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:39.593229 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b091ef9-ed86-41dc-a6ec-651eb67dcbe9 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:39.593734 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b091ef9-ed86-41dc-a6ec-651eb67dcbe9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:39.593879 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-c8c89f1a-121a-4805-906c-6e4daf74862b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '86ecc69eb95b4533b29d710b5c6c804c'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:39.615207 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-2b091ef9-ed86-41dc-a6ec-651eb67dcbe9 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:26:39.616457 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-2b091ef9-ed86-41dc-a6ec-651eb67dcbe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:26:39.623174 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-c8c89f1a-121a-4805-906c-6e4daf74862b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '86ecc69eb95b4533b29d710b5c6c804c'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:39.623815 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-c8c89f1a-121a-4805-906c-6e4daf74862b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '86ecc69eb95b4533b29d710b5c6c804c'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:39.624782 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1033/2060] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:26:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:26:39.627951 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b091ef9-ed86-41dc-a6ec-651eb67dcbe9 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:39.627951 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-2b091ef9-ed86-41dc-a6ec-651eb67dcbe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:26:39.637407 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1028/2061] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jun 29 11:26:38 2022] DELETE /identity/v3/users/86ecc69eb95b4533b29d710b5c6c804c => generated 0 bytes in 718 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:26:39.638269 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c89f1a-121a-4805-906c-6e4daf74862b admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:39.639500 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b091ef9-ed86-41dc-a6ec-651eb67dcbe9 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:39.639956 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-c8c89f1a-121a-4805-906c-6e4daf74862b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:26:39.675673 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=84,keystone:INSERT=3,keystone:DELETE=13 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:26:39.696079 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604f16b9-d15f-49e8-aa4d-086a48ec2c31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:39.696753 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604f16b9-d15f-49e8-aa4d-086a48ec2c31 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:39.697048 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604f16b9-d15f-49e8-aa4d-086a48ec2c31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:39.697312 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604f16b9-d15f-49e8-aa4d-086a48ec2c31 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:39.698077 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c89f1a-121a-4805-906c-6e4daf74862b admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:39.698182 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604f16b9-d15f-49e8-aa4d-086a48ec2c31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:39.699785 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604f16b9-d15f-49e8-aa4d-086a48ec2c31 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:39.701958 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdd3536-9152-4fc5-af10-db14c99bcd01 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f4bb9f989e4024a5da28dced646b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:39.702666 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdd3536-9152-4fc5-af10-db14c99bcd01 service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:39.702956 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdd3536-9152-4fc5-af10-db14c99bcd01 service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:39.711392 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdd3536-9152-4fc5-af10-db14c99bcd01 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:39.711562 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1034/2062] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:26:39 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 11:26:39.711737 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91a4b15-017f-4740-8a34-3f7d0504ac67 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:39.712054 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91a4b15-017f-4740-8a34-3f7d0504ac67 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:39.712326 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91a4b15-017f-4740-8a34-3f7d0504ac67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:39.716688 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdd3536-9152-4fc5-af10-db14c99bcd01 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:39.718163 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdd3536-9152-4fc5-af10-db14c99bcd01 service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:39.728103 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1029/2063] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:26:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:26:39.735138 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdd3536-9152-4fc5-af10-db14c99bcd01 service neutron] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:39.735401 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-f91a4b15-017f-4740-8a34-3f7d0504ac67 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:26:39.735401 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-f91a4b15-017f-4740-8a34-3f7d0504ac67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:26:39.743159 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdd3536-9152-4fc5-af10-db14c99bcd01 service neutron] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:39.743423 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1035/2064] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:26:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:26:39.748823 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fe84b2-6ec0-4880-8b01-fb9421088bf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:39.749855 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fe84b2-6ec0-4880-8b01-fb9421088bf8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:39.750140 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fe84b2-6ec0-4880-8b01-fb9421088bf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:39.750480 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fe84b2-6ec0-4880-8b01-fb9421088bf8 admin admin] PATH_INFO: `/v3/projects/35f8b66a4a8f472ab1e8928cdd3fb6cd/users/0303383b06be4fb0b976f194194919d0/roles/e1c88eebe911432cab6a07fbeb84653b` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:39.755411 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91a4b15-017f-4740-8a34-3f7d0504ac67 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:39.760119 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-f91a4b15-017f-4740-8a34-3f7d0504ac67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:26:39.773845 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fe84b2-6ec0-4880-8b01-fb9421088bf8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=35f8b66a4a8f472ab1e8928cdd3fb6cd, user_id=0303383b06be4fb0b976f194194919d0, role_id=e1c88eebe911432cab6a07fbeb84653b)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:39.776458 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91a4b15-017f-4740-8a34-3f7d0504ac67 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:39.776676 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fe84b2-6ec0-4880-8b01-fb9421088bf8 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:39.787472 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d87eda3-abf8-43bd-8857-e2a6f321b348 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f4bb9f989e4024a5da28dced646b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:39.791756 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d87eda3-abf8-43bd-8857-e2a6f321b348 service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:39.791756 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d87eda3-abf8-43bd-8857-e2a6f321b348 service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:39.791756 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d87eda3-abf8-43bd-8857-e2a6f321b348 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:39.806700 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d87eda3-abf8-43bd-8857-e2a6f321b348 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:39.806700 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d87eda3-abf8-43bd-8857-e2a6f321b348 service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:39.810956 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:26:39.819886 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1036/2065] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:26:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:26:39.849077 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1030/2066] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jun 29 11:26:39 2022] PUT /identity/v3/projects/35f8b66a4a8f472ab1e8928cdd3fb6cd/users/0303383b06be4fb0b976f194194919d0/roles/e1c88eebe911432cab6a07fbeb84653b => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:26:39.862322 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae6684e-85da-4219-94c7-6ee4399fef88 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:39.862621 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae6684e-85da-4219-94c7-6ee4399fef88 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:39.862903 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae6684e-85da-4219-94c7-6ee4399fef88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:39.892995 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-5ae6684e-85da-4219-94c7-6ee4399fef88 None None] MFA Rules not processed for user `0303383b06be4fb0b976f194194919d0`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:26:39.894590 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-5ae6684e-85da-4219-94c7-6ee4399fef88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:26:39.984665 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1037/2067] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:26:39 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:26:39.998216 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fe84b2-6ec0-4880-8b01-fb9421088bf8 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:39.999972 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-14fe84b2-6ec0-4880-8b01-fb9421088bf8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:26:40.109171 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fe84b2-6ec0-4880-8b01-fb9421088bf8 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:40.119031 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7498f376-ac72-4049-9581-fe6530617278 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede3a17c7063432e8ee66462766584f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:40.119031 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7498f376-ac72-4049-9581-fe6530617278 service glance] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:40.119031 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7498f376-ac72-4049-9581-fe6530617278 service glance] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:40.119031 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7498f376-ac72-4049-9581-fe6530617278 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:40.147235 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7498f376-ac72-4049-9581-fe6530617278 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:40.147235 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7498f376-ac72-4049-9581-fe6530617278 service glance] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:40.153895 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1031/2068] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:26:39 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 11:26:40.337852 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae6684e-85da-4219-94c7-6ee4399fef88 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:40.339966 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-5ae6684e-85da-4219-94c7-6ee4399fef88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:26:40.368868 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7498f376-ac72-4049-9581-fe6530617278 service glance] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:40.370352 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-7498f376-ac72-4049-9581-fe6530617278 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:26:40.430267 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae6684e-85da-4219-94c7-6ee4399fef88 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:40.435871 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6a66b6-75a2-4a48-a5ad-ef29648ebafe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:40.439615 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6a66b6-75a2-4a48-a5ad-ef29648ebafe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:40.439615 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6a66b6-75a2-4a48-a5ad-ef29648ebafe admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:40.439615 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6a66b6-75a2-4a48-a5ad-ef29648ebafe admin admin] PATH_INFO: `/v3/projects/2c8b9fe1a01c4cd9a44f6e6250a1d0de` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:40.439615 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6a66b6-75a2-4a48-a5ad-ef29648ebafe admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c8b9fe1a01c4cd9a44f6e6250a1d0de)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:40.440971 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6a66b6-75a2-4a48-a5ad-ef29648ebafe admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:40.468365 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7498f376-ac72-4049-9581-fe6530617278 service glance] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:40.473653 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbbb470-f376-4414-838d-9be80b4370fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:40.474527 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbbb470-f376-4414-838d-9be80b4370fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:40.474814 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbbb470-f376-4414-838d-9be80b4370fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:40.475092 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbbb470-f376-4414-838d-9be80b4370fd admin admin] PATH_INFO: `/v3/projects/4544fce0292645b481f7939f6f4ac05a` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:40.476326 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbbb470-f376-4414-838d-9be80b4370fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=4544fce0292645b481f7939f6f4ac05a)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:40.478242 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbbb470-f376-4414-838d-9be80b4370fd admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:40.623477 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-6c6a66b6-75a2-4a48-a5ad-ef29648ebafe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c8b9fe1a01c4cd9a44f6e6250a1d0de', 'request_id': 'req-6c6a66b6-75a2-4a48-a5ad-ef29648ebafe'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:40.624180 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-1fbbb470-f376-4414-838d-9be80b4370fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4544fce0292645b481f7939f6f4ac05a', 'request_id': 'req-1fbbb470-f376-4414-838d-9be80b4370fd'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:40.629156 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-6c6a66b6-75a2-4a48-a5ad-ef29648ebafe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c8b9fe1a01c4cd9a44f6e6250a1d0de', 'request_id': 'req-6c6a66b6-75a2-4a48-a5ad-ef29648ebafe'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:40.635393 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-6c6a66b6-75a2-4a48-a5ad-ef29648ebafe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c8b9fe1a01c4cd9a44f6e6250a1d0de', 'request_id': 'req-6c6a66b6-75a2-4a48-a5ad-ef29648ebafe'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:40.738863 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-1fbbb470-f376-4414-838d-9be80b4370fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4544fce0292645b481f7939f6f4ac05a', 'request_id': 'req-1fbbb470-f376-4414-838d-9be80b4370fd'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:40.745405 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-1fbbb470-f376-4414-838d-9be80b4370fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4544fce0292645b481f7939f6f4ac05a', 'request_id': 'req-1fbbb470-f376-4414-838d-9be80b4370fd'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:40.756074 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-1fbbb470-f376-4414-838d-9be80b4370fd admin admin] The token cache is being invalidate because project 4544fce0292645b481f7939f6f4ac05a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 11:26:40.756374 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-1fbbb470-f376-4414-838d-9be80b4370fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:40.757056 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-1fbbb470-f376-4414-838d-9be80b4370fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:40.758652 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1032/2069] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jun 29 11:26:40 2022] DELETE /identity/v3/projects/4544fce0292645b481f7939f6f4ac05a => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:26:40.762629 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbbb470-f376-4414-838d-9be80b4370fd admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:40.764492 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-1fbbb470-f376-4414-838d-9be80b4370fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:26:40.827393 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbbb470-f376-4414-838d-9be80b4370fd admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:40.827574 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-6c6a66b6-75a2-4a48-a5ad-ef29648ebafe admin admin] The token cache is being invalidate because project 2c8b9fe1a01c4cd9a44f6e6250a1d0de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 11:26:40.828015 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-6c6a66b6-75a2-4a48-a5ad-ef29648ebafe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:40.828794 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-6c6a66b6-75a2-4a48-a5ad-ef29648ebafe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:40.830746 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1038/2070] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jun 29 11:26:40 2022] DELETE /identity/v3/projects/2c8b9fe1a01c4cd9a44f6e6250a1d0de => generated 0 bytes in 493 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:26:40.835348 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c61fa91-f97b-436d-96eb-94c76a99f6e2 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:40.835591 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c61fa91-f97b-436d-96eb-94c76a99f6e2 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:40.835910 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c61fa91-f97b-436d-96eb-94c76a99f6e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:40.856983 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-4c61fa91-f97b-436d-96eb-94c76a99f6e2 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:26:40.858321 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-4c61fa91-f97b-436d-96eb-94c76a99f6e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:26:40.884825 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee91a3e-4fa4-4dd3-aa2b-d5a090054bc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:40.885692 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee91a3e-4fa4-4dd3-aa2b-d5a090054bc8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:40.889218 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee91a3e-4fa4-4dd3-aa2b-d5a090054bc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:40.889218 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee91a3e-4fa4-4dd3-aa2b-d5a090054bc8 admin admin] PATH_INFO: `/v3/users/883f1daa527b4acf8c4d15010ac405c0` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:40.889218 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee91a3e-4fa4-4dd3-aa2b-d5a090054bc8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=883f1daa527b4acf8c4d15010ac405c0)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:40.890295 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee91a3e-4fa4-4dd3-aa2b-d5a090054bc8 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:40.921371 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1039/2071] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:26:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 11:26:40.929929 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25ad323-b9dc-484b-8db2-4a0bd9a19144 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:40.930848 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25ad323-b9dc-484b-8db2-4a0bd9a19144 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:40.931144 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25ad323-b9dc-484b-8db2-4a0bd9a19144 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:40.952158 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-b25ad323-b9dc-484b-8db2-4a0bd9a19144 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:26:40.953410 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-b25ad323-b9dc-484b-8db2-4a0bd9a19144 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:26:40.961428 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1040/2072] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:26:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 11:26:40.965438 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25ad323-b9dc-484b-8db2-4a0bd9a19144 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:40.974986 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25ad323-b9dc-484b-8db2-4a0bd9a19144 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:40.979791 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dcbac9-aaa4-4555-b611-8cac4dfa2dbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:40.980489 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dcbac9-aaa4-4555-b611-8cac4dfa2dbf admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:40.980771 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dcbac9-aaa4-4555-b611-8cac4dfa2dbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:40.981046 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dcbac9-aaa4-4555-b611-8cac4dfa2dbf admin admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:40.984031 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dcbac9-aaa4-4555-b611-8cac4dfa2dbf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:26:40.984031 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dcbac9-aaa4-4555-b611-8cac4dfa2dbf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:40.984031 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dcbac9-aaa4-4555-b611-8cac4dfa2dbf admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:40.994967 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1041/2073] 10.222.0.44 () {58 vars in 1152 bytes} [Wed Jun 29 11:26:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 11:26:40.999501 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dcbac9-aaa4-4555-b611-8cac4dfa2dbf admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:41.008678 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dcbac9-aaa4-4555-b611-8cac4dfa2dbf admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:41.013192 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef22862c-75ae-4193-9e97-3f4882eafcc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:41.014438 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef22862c-75ae-4193-9e97-3f4882eafcc3 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:41.014722 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef22862c-75ae-4193-9e97-3f4882eafcc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:41.015853 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef22862c-75ae-4193-9e97-3f4882eafcc3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:41.015853 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef22862c-75ae-4193-9e97-3f4882eafcc3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:41.017524 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef22862c-75ae-4193-9e97-3f4882eafcc3 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:41.131454 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1042/2074] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jun 29 11:26:40 2022] POST /identity/v3/projects => generated 385 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 11:26:41.135598 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef22862c-75ae-4193-9e97-3f4882eafcc3 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:41.198026 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-8ee91a3e-4fa4-4dd3-aa2b-d5a090054bc8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '883f1daa527b4acf8c4d15010ac405c0'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:41.234606 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef22862c-75ae-4193-9e97-3f4882eafcc3 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:41.241594 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c50220-8a91-47cd-89c4-6a0c18ec1aec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:41.241594 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c50220-8a91-47cd-89c4-6a0c18ec1aec admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:41.241594 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c50220-8a91-47cd-89c4-6a0c18ec1aec admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:41.241961 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c50220-8a91-47cd-89c4-6a0c18ec1aec admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:41.242398 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c50220-8a91-47cd-89c4-6a0c18ec1aec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:41.244488 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c50220-8a91-47cd-89c4-6a0c18ec1aec admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:41.307358 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-8ee91a3e-4fa4-4dd3-aa2b-d5a090054bc8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '883f1daa527b4acf8c4d15010ac405c0'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:41.308010 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-8ee91a3e-4fa4-4dd3-aa2b-d5a090054bc8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '883f1daa527b4acf8c4d15010ac405c0'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:41.320713 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-8ee91a3e-4fa4-4dd3-aa2b-d5a090054bc8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '883f1daa527b4acf8c4d15010ac405c0'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:41.321365 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-8ee91a3e-4fa4-4dd3-aa2b-d5a090054bc8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '883f1daa527b4acf8c4d15010ac405c0'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:41.333339 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1033/2075] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jun 29 11:26:40 2022] DELETE /identity/v3/users/883f1daa527b4acf8c4d15010ac405c0 => generated 0 bytes in 571 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 11:26:41.343018 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:26:41.348957 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1043/2076] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jun 29 11:26:41 2022] POST /identity/v3/users => generated 293 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 11:26:41.353847 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee91a3e-4fa4-4dd3-aa2b-d5a090054bc8 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:41.355316 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-8ee91a3e-4fa4-4dd3-aa2b-d5a090054bc8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:26:41.433930 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee91a3e-4fa4-4dd3-aa2b-d5a090054bc8 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:41.439044 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:26:41.444383 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6da059-7178-49ba-8774-805ca252f488 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:41.445102 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6da059-7178-49ba-8774-805ca252f488 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:41.445390 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6da059-7178-49ba-8774-805ca252f488 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:41.445667 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6da059-7178-49ba-8774-805ca252f488 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:41.446439 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6da059-7178-49ba-8774-805ca252f488 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:41.448121 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6da059-7178-49ba-8774-805ca252f488 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:41.459264 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1034/2077] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:26:41 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 11:26:41.465852 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c50220-8a91-47cd-89c4-6a0c18ec1aec admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:41.476689 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c50220-8a91-47cd-89c4-6a0c18ec1aec admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:41.481222 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4776dd82-99ce-418d-be1a-449ba0a6d243 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:41.481880 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4776dd82-99ce-418d-be1a-449ba0a6d243 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:41.482195 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4776dd82-99ce-418d-be1a-449ba0a6d243 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:41.482473 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4776dd82-99ce-418d-be1a-449ba0a6d243 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:41.483201 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4776dd82-99ce-418d-be1a-449ba0a6d243 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:41.484883 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4776dd82-99ce-418d-be1a-449ba0a6d243 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:41.494856 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1044/2078] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:26:41 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 11:26:41.501504 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6da059-7178-49ba-8774-805ca252f488 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:41.509376 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6da059-7178-49ba-8774-805ca252f488 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:41.514515 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1342950-6c51-4b38-a167-5e36e22b9278 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:41.515377 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1342950-6c51-4b38-a167-5e36e22b9278 admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:41.515655 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1342950-6c51-4b38-a167-5e36e22b9278 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:41.515962 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1342950-6c51-4b38-a167-5e36e22b9278 admin admin] PATH_INFO: `/v3/projects/3eefc925452241a0a4642d1ef46245a9/users/5ffed2ecca0c42fda7befa7a0934101c/roles/e1c88eebe911432cab6a07fbeb84653b` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:41.532608 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1342950-6c51-4b38-a167-5e36e22b9278 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3eefc925452241a0a4642d1ef46245a9, user_id=5ffed2ecca0c42fda7befa7a0934101c, role_id=e1c88eebe911432cab6a07fbeb84653b)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:41.535297 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1342950-6c51-4b38-a167-5e36e22b9278 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:41.649247 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1035/2079] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jun 29 11:26:41 2022] PUT /identity/v3/projects/3eefc925452241a0a4642d1ef46245a9/users/5ffed2ecca0c42fda7befa7a0934101c/roles/e1c88eebe911432cab6a07fbeb84653b => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:26:41.659818 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f572a02-f7d1-4993-881c-5ebe3ac77844 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:41.660303 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f572a02-f7d1-4993-881c-5ebe3ac77844 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:41.660534 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f572a02-f7d1-4993-881c-5ebe3ac77844 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:41.692177 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-4f572a02-f7d1-4993-881c-5ebe3ac77844 None None] MFA Rules not processed for user `5ffed2ecca0c42fda7befa7a0934101c`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:26:41.693429 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-4f572a02-f7d1-4993-881c-5ebe3ac77844 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:26:41.767176 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1045/2080] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:26:41 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:26:41.771730 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1342950-6c51-4b38-a167-5e36e22b9278 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:41.791409 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f572a02-f7d1-4993-881c-5ebe3ac77844 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:41.793105 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-4f572a02-f7d1-4993-881c-5ebe3ac77844 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:26:41.838100 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1342950-6c51-4b38-a167-5e36e22b9278 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:41.843200 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4916bbf8-b7ec-4e48-be23-ae07c0c24e6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:41.843956 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4916bbf8-b7ec-4e48-be23-ae07c0c24e6a admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:41.844244 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4916bbf8-b7ec-4e48-be23-ae07c0c24e6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:41.844519 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4916bbf8-b7ec-4e48-be23-ae07c0c24e6a admin admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:41.845512 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4916bbf8-b7ec-4e48-be23-ae07c0c24e6a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:41.848470 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4916bbf8-b7ec-4e48-be23-ae07c0c24e6a admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:41.866964 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f572a02-f7d1-4993-881c-5ebe3ac77844 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:41.871595 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7042a059-860a-4c18-a8d0-6f2f00587426 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:41.872472 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7042a059-860a-4c18-a8d0-6f2f00587426 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:41.873275 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7042a059-860a-4c18-a8d0-6f2f00587426 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:41.873556 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7042a059-860a-4c18-a8d0-6f2f00587426 admin admin] PATH_INFO: `/v3/projects/3061d9b3ca8a4a3988dfc8c4578c7cf0` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:41.874720 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7042a059-860a-4c18-a8d0-6f2f00587426 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3061d9b3ca8a4a3988dfc8c4578c7cf0)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:41.876644 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7042a059-860a-4c18-a8d0-6f2f00587426 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:41.928659 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1036/2081] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jun 29 11:26:41 2022] POST /identity/v3/projects => generated 385 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:26:41.932572 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4916bbf8-b7ec-4e48-be23-ae07c0c24e6a admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:41.991856 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4916bbf8-b7ec-4e48-be23-ae07c0c24e6a admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:42.004828 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-7042a059-860a-4c18-a8d0-6f2f00587426 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3061d9b3ca8a4a3988dfc8c4578c7cf0', 'request_id': 'req-7042a059-860a-4c18-a8d0-6f2f00587426'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:42.011037 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-7042a059-860a-4c18-a8d0-6f2f00587426 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3061d9b3ca8a4a3988dfc8c4578c7cf0', 'request_id': 'req-7042a059-860a-4c18-a8d0-6f2f00587426'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:42.017048 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-7042a059-860a-4c18-a8d0-6f2f00587426 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3061d9b3ca8a4a3988dfc8c4578c7cf0', 'request_id': 'req-7042a059-860a-4c18-a8d0-6f2f00587426'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:42.046500 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8077d2-8c89-41f0-bc57-c0588079df80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:42.047124 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8077d2-8c89-41f0-bc57-c0588079df80 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:42.047410 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8077d2-8c89-41f0-bc57-c0588079df80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:42.047683 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8077d2-8c89-41f0-bc57-c0588079df80 admin admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:42.048411 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8077d2-8c89-41f0-bc57-c0588079df80 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:42.050319 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8077d2-8c89-41f0-bc57-c0588079df80 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:42.093547 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:26:42.112593 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-7042a059-860a-4c18-a8d0-6f2f00587426 admin admin] The token cache is being invalidate because project 3061d9b3ca8a4a3988dfc8c4578c7cf0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 11:26:42.112886 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-7042a059-860a-4c18-a8d0-6f2f00587426 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:42.113525 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-7042a059-860a-4c18-a8d0-6f2f00587426 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:42.115076 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1046/2082] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jun 29 11:26:41 2022] DELETE /identity/v3/projects/3061d9b3ca8a4a3988dfc8c4578c7cf0 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 11:26:42.119764 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a622ca-ea44-4c74-8db9-ad74e2224236 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:42.120078 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a622ca-ea44-4c74-8db9-ad74e2224236 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:42.120361 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a622ca-ea44-4c74-8db9-ad74e2224236 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:42.138440 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-f5a622ca-ea44-4c74-8db9-ad74e2224236 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:26:42.140237 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-f5a622ca-ea44-4c74-8db9-ad74e2224236 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:26:42.148375 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1047/2083] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:26:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 11:26:42.152981 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1037/2084] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jun 29 11:26:41 2022] POST /identity/v3/users => generated 293 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 11:26:42.155969 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa896dc-aef9-4c1a-ab32-da3865c861ea None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:42.157560 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa896dc-aef9-4c1a-ab32-da3865c861ea None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:42.157560 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa896dc-aef9-4c1a-ab32-da3865c861ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:42.160458 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8077d2-8c89-41f0-bc57-c0588079df80 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:42.162024 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-8c8077d2-8c89-41f0-bc57-c0588079df80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:26:42.177375 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-4fa896dc-aef9-4c1a-ab32-da3865c861ea None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:26:42.177792 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8077d2-8c89-41f0-bc57-c0588079df80 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:42.178659 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-4fa896dc-aef9-4c1a-ab32-da3865c861ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:26:42.183934 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bba111-8c6b-458a-b69f-a1dc34aeebdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:42.184803 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bba111-8c6b-458a-b69f-a1dc34aeebdf admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:42.186203 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bba111-8c6b-458a-b69f-a1dc34aeebdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:42.186551 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bba111-8c6b-458a-b69f-a1dc34aeebdf admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:42.187373 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bba111-8c6b-458a-b69f-a1dc34aeebdf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:42.190033 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bba111-8c6b-458a-b69f-a1dc34aeebdf admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:42.192171 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1048/2085] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:26:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:26:42.199399 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa896dc-aef9-4c1a-ab32-da3865c861ea None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:42.208209 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa896dc-aef9-4c1a-ab32-da3865c861ea None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:42.210351 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1038/2086] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:26:42 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 11:26:42.213394 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8439791c-8295-43b4-9d49-164dd7dec96a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:42.215640 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bba111-8c6b-458a-b69f-a1dc34aeebdf admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:42.215757 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8439791c-8295-43b4-9d49-164dd7dec96a admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:42.215757 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8439791c-8295-43b4-9d49-164dd7dec96a admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:42.215757 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8439791c-8295-43b4-9d49-164dd7dec96a admin admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:42.215757 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8439791c-8295-43b4-9d49-164dd7dec96a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:26:42.216137 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8439791c-8295-43b4-9d49-164dd7dec96a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:42.217542 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8439791c-8295-43b4-9d49-164dd7dec96a admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:42.226470 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bba111-8c6b-458a-b69f-a1dc34aeebdf admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:42.231848 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364c2977-8ddf-41e5-ac56-79efe306f859 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:42.233109 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364c2977-8ddf-41e5-ac56-79efe306f859 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:42.233399 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364c2977-8ddf-41e5-ac56-79efe306f859 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:42.233673 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364c2977-8ddf-41e5-ac56-79efe306f859 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:42.234405 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364c2977-8ddf-41e5-ac56-79efe306f859 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:42.236106 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364c2977-8ddf-41e5-ac56-79efe306f859 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:42.240669 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1049/2087] 10.222.0.44 () {58 vars in 1152 bytes} [Wed Jun 29 11:26:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 11:26:42.243632 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:INSERT=2,keystone:DELETE=13,keystone:SELECT=85 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:26:42.245248 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8439791c-8295-43b4-9d49-164dd7dec96a admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:42.253040 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1039/2088] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:26:42 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 11:26:42.254041 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364c2977-8ddf-41e5-ac56-79efe306f859 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:42.256652 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8439791c-8295-43b4-9d49-164dd7dec96a admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:42.262185 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f6a02d-23ae-4df2-b837-97398b44d530 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:42.262853 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f6a02d-23ae-4df2-b837-97398b44d530 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:42.263187 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f6a02d-23ae-4df2-b837-97398b44d530 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:42.263496 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f6a02d-23ae-4df2-b837-97398b44d530 admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:42.264225 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f6a02d-23ae-4df2-b837-97398b44d530 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:42.266108 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f6a02d-23ae-4df2-b837-97398b44d530 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:42.266792 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364c2977-8ddf-41e5-ac56-79efe306f859 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:42.279201 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a45296-e2b7-4dfb-8d00-e1279a74ec26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:42.279201 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a45296-e2b7-4dfb-8d00-e1279a74ec26 admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:42.279201 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a45296-e2b7-4dfb-8d00-e1279a74ec26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:42.279201 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a45296-e2b7-4dfb-8d00-e1279a74ec26 admin admin] PATH_INFO: `/v3/projects/e2719678466341ab92f6ee2bee3ae3f2/users/ca65bb764a1841e6acc1f7feeea98a26/roles/e1c88eebe911432cab6a07fbeb84653b` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:42.288916 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a45296-e2b7-4dfb-8d00-e1279a74ec26 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2719678466341ab92f6ee2bee3ae3f2, user_id=ca65bb764a1841e6acc1f7feeea98a26, role_id=e1c88eebe911432cab6a07fbeb84653b)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:42.291557 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a45296-e2b7-4dfb-8d00-e1279a74ec26 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:42.346043 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1050/2089] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jun 29 11:26:42 2022] POST /identity/v3/projects => generated 369 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:26:42.350078 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f6a02d-23ae-4df2-b837-97398b44d530 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:42.380559 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1040/2090] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jun 29 11:26:42 2022] PUT /identity/v3/projects/e2719678466341ab92f6ee2bee3ae3f2/users/ca65bb764a1841e6acc1f7feeea98a26/roles/e1c88eebe911432cab6a07fbeb84653b => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:26:42.390611 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd96af7-9b77-45cb-b5aa-3b9baab4a01b None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:42.391013 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd96af7-9b77-45cb-b5aa-3b9baab4a01b None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:42.391358 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd96af7-9b77-45cb-b5aa-3b9baab4a01b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:42.411168 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f6a02d-23ae-4df2-b837-97398b44d530 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:42.418383 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-3bd96af7-9b77-45cb-b5aa-3b9baab4a01b None None] MFA Rules not processed for user `ca65bb764a1841e6acc1f7feeea98a26`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:26:42.420158 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-3bd96af7-9b77-45cb-b5aa-3b9baab4a01b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:26:42.471058 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0ec055-0fb5-40e8-9e2e-e4e1f149ab88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:42.471922 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0ec055-0fb5-40e8-9e2e-e4e1f149ab88 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:42.472304 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0ec055-0fb5-40e8-9e2e-e4e1f149ab88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:42.472700 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0ec055-0fb5-40e8-9e2e-e4e1f149ab88 admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:42.473474 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0ec055-0fb5-40e8-9e2e-e4e1f149ab88 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:42.475691 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0ec055-0fb5-40e8-9e2e-e4e1f149ab88 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:42.486654 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1041/2091] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:26:42 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 11:26:42.491040 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd96af7-9b77-45cb-b5aa-3b9baab4a01b None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:42.501789 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd96af7-9b77-45cb-b5aa-3b9baab4a01b None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:42.506330 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900c6605-ca87-44c8-8de2-45ebef1dee00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:42.506955 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900c6605-ca87-44c8-8de2-45ebef1dee00 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:42.507234 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900c6605-ca87-44c8-8de2-45ebef1dee00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:42.507501 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900c6605-ca87-44c8-8de2-45ebef1dee00 admin admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:42.508210 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900c6605-ca87-44c8-8de2-45ebef1dee00 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:42.510129 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900c6605-ca87-44c8-8de2-45ebef1dee00 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:42.608157 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1051/2092] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jun 29 11:26:42 2022] POST /identity/v3/users => generated 285 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 11:26:42.612026 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0ec055-0fb5-40e8-9e2e-e4e1f149ab88 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:42.621737 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0ec055-0fb5-40e8-9e2e-e4e1f149ab88 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:42.626385 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7b09c8-10b4-4a08-bf3b-0c2c414ed5bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:42.627030 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7b09c8-10b4-4a08-bf3b-0c2c414ed5bc admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:42.627314 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7b09c8-10b4-4a08-bf3b-0c2c414ed5bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:42.627591 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7b09c8-10b4-4a08-bf3b-0c2c414ed5bc admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:42.628387 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7b09c8-10b4-4a08-bf3b-0c2c414ed5bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:42.630353 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7b09c8-10b4-4a08-bf3b-0c2c414ed5bc admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:42.638225 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1042/2093] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jun 29 11:26:42 2022] POST /identity/v3/projects => generated 383 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:26:42.642007 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1052/2094] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:26:42 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 11:26:42.645039 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900c6605-ca87-44c8-8de2-45ebef1dee00 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:42.645193 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7b09c8-10b4-4a08-bf3b-0c2c414ed5bc admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:42.656102 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7b09c8-10b4-4a08-bf3b-0c2c414ed5bc admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:42.704962 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900c6605-ca87-44c8-8de2-45ebef1dee00 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:42.709854 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fed6cfc-5dd4-416d-87df-d1e8aa1a9bc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:42.710526 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fed6cfc-5dd4-416d-87df-d1e8aa1a9bc0 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:42.710808 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fed6cfc-5dd4-416d-87df-d1e8aa1a9bc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:42.711087 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fed6cfc-5dd4-416d-87df-d1e8aa1a9bc0 admin admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:42.711777 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fed6cfc-5dd4-416d-87df-d1e8aa1a9bc0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:42.713683 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c795fc71-7858-4fcd-adb2-d78c6cb52408 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:42.714097 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fed6cfc-5dd4-416d-87df-d1e8aa1a9bc0 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:42.714397 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c795fc71-7858-4fcd-adb2-d78c6cb52408 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:42.714680 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c795fc71-7858-4fcd-adb2-d78c6cb52408 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:42.714957 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c795fc71-7858-4fcd-adb2-d78c6cb52408 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:42.715669 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c795fc71-7858-4fcd-adb2-d78c6cb52408 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:42.717874 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c795fc71-7858-4fcd-adb2-d78c6cb52408 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:42.728111 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1053/2095] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:26:42 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 11:26:42.731928 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c795fc71-7858-4fcd-adb2-d78c6cb52408 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:42.742028 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c795fc71-7858-4fcd-adb2-d78c6cb52408 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:42.746764 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0964006-d81b-478c-a203-b7a6c5bd8bad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:42.747626 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0964006-d81b-478c-a203-b7a6c5bd8bad admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:42.747967 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0964006-d81b-478c-a203-b7a6c5bd8bad admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:42.748245 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0964006-d81b-478c-a203-b7a6c5bd8bad admin admin] PATH_INFO: `/v3/projects/a5ad5c579bbd42a3ad80cbbd556f5b0b/users/594ddbf92764482eac3b5dc20464b3f4/roles/e1c88eebe911432cab6a07fbeb84653b` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:42.762927 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0964006-d81b-478c-a203-b7a6c5bd8bad admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5ad5c579bbd42a3ad80cbbd556f5b0b, user_id=594ddbf92764482eac3b5dc20464b3f4, role_id=e1c88eebe911432cab6a07fbeb84653b)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:42.765740 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0964006-d81b-478c-a203-b7a6c5bd8bad admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:42.810371 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1043/2096] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jun 29 11:26:42 2022] POST /identity/v3/users => generated 298 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:26:42.814300 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fed6cfc-5dd4-416d-87df-d1e8aa1a9bc0 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:42.823838 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fed6cfc-5dd4-416d-87df-d1e8aa1a9bc0 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:42.828626 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30416914-2d99-4231-aad8-d7ddd628f77c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:42.829288 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30416914-2d99-4231-aad8-d7ddd628f77c admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:42.829595 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30416914-2d99-4231-aad8-d7ddd628f77c admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:42.829869 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30416914-2d99-4231-aad8-d7ddd628f77c admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:42.830578 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30416914-2d99-4231-aad8-d7ddd628f77c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:42.832264 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30416914-2d99-4231-aad8-d7ddd628f77c admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:42.842418 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1044/2097] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:26:42 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 11:26:42.846657 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30416914-2d99-4231-aad8-d7ddd628f77c admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:42.855729 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30416914-2d99-4231-aad8-d7ddd628f77c admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:42.860977 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e01511e-bcf8-4d08-ad8d-abc410fde8dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:42.861748 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e01511e-bcf8-4d08-ad8d-abc410fde8dd admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:42.862025 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e01511e-bcf8-4d08-ad8d-abc410fde8dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:42.862297 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e01511e-bcf8-4d08-ad8d-abc410fde8dd admin admin] PATH_INFO: `/v3/projects/4657d8718c8f4ef2890100d9b33cf31e/users/ba278ccf74424f729568a18219d92f8d/roles/e95c2ce63e114cc19a9dda6ad21e794f` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:42.866996 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1054/2098] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jun 29 11:26:42 2022] PUT /identity/v3/projects/a5ad5c579bbd42a3ad80cbbd556f5b0b/users/594ddbf92764482eac3b5dc20464b3f4/roles/e1c88eebe911432cab6a07fbeb84653b => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 11:26:42.878161 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e01511e-bcf8-4d08-ad8d-abc410fde8dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=4657d8718c8f4ef2890100d9b33cf31e, user_id=ba278ccf74424f729568a18219d92f8d, role_id=e95c2ce63e114cc19a9dda6ad21e794f)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:42.880650 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12553bb5-7295-458d-a786-2b1d0dc96beb None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:42.880900 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e01511e-bcf8-4d08-ad8d-abc410fde8dd admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:42.880997 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12553bb5-7295-458d-a786-2b1d0dc96beb None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:42.881209 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12553bb5-7295-458d-a786-2b1d0dc96beb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:42.890576 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:26:42.909642 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-12553bb5-7295-458d-a786-2b1d0dc96beb None None] MFA Rules not processed for user `594ddbf92764482eac3b5dc20464b3f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:26:42.910859 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-12553bb5-7295-458d-a786-2b1d0dc96beb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:26:42.953182 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1045/2099] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jun 29 11:26:42 2022] PUT /identity/v3/projects/4657d8718c8f4ef2890100d9b33cf31e/users/ba278ccf74424f729568a18219d92f8d/roles/e95c2ce63e114cc19a9dda6ad21e794f => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 11:26:42.957129 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e01511e-bcf8-4d08-ad8d-abc410fde8dd admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:42.974624 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1055/2100] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:26:42 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 11:26:42.978873 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12553bb5-7295-458d-a786-2b1d0dc96beb None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:42.985504 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:26:43.019668 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e01511e-bcf8-4d08-ad8d-abc410fde8dd admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:43.025611 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a65872-39e3-4536-8343-c439c2699896 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:43.026270 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a65872-39e3-4536-8343-c439c2699896 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:43.026556 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a65872-39e3-4536-8343-c439c2699896 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:43.026829 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a65872-39e3-4536-8343-c439c2699896 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:43.027557 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a65872-39e3-4536-8343-c439c2699896 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:43.029765 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a65872-39e3-4536-8343-c439c2699896 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:43.039493 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1046/2101] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:26:42 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 11:26:43.040804 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12553bb5-7295-458d-a786-2b1d0dc96beb None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:43.043512 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a65872-39e3-4536-8343-c439c2699896 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:43.045557 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dca526-d5bd-44ff-8821-fbb64336923b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:43.046221 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dca526-d5bd-44ff-8821-fbb64336923b admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:43.046508 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dca526-d5bd-44ff-8821-fbb64336923b admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:43.046790 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dca526-d5bd-44ff-8821-fbb64336923b admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:43.047482 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3dca526-d5bd-44ff-8821-fbb64336923b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:43.049530 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3dca526-d5bd-44ff-8821-fbb64336923b admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:43.052893 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a65872-39e3-4536-8343-c439c2699896 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:43.057554 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab35c4eb-52c6-4bf2-ab48-d9f2a9132a42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:43.059085 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab35c4eb-52c6-4bf2-ab48-d9f2a9132a42 admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:43.059427 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab35c4eb-52c6-4bf2-ab48-d9f2a9132a42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:43.059703 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab35c4eb-52c6-4bf2-ab48-d9f2a9132a42 admin admin] PATH_INFO: `/v3/projects/4657d8718c8f4ef2890100d9b33cf31e/users/ba278ccf74424f729568a18219d92f8d/roles/e95c2ce63e114cc19a9dda6ad21e794f` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:43.061774 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab35c4eb-52c6-4bf2-ab48-d9f2a9132a42 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4657d8718c8f4ef2890100d9b33cf31e, user_id=ba278ccf74424f729568a18219d92f8d, role_id=e95c2ce63e114cc19a9dda6ad21e794f)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:43.064408 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab35c4eb-52c6-4bf2-ab48-d9f2a9132a42 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:43.074512 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1047/2102] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jun 29 11:26:43 2022] PUT /identity/v3/projects/4657d8718c8f4ef2890100d9b33cf31e/users/ba278ccf74424f729568a18219d92f8d/roles/e95c2ce63e114cc19a9dda6ad21e794f => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 11:26:43.086025 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dd247c-2a31-452e-8b8e-00d146dc8aa0 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:43.086315 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dd247c-2a31-452e-8b8e-00d146dc8aa0 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:43.086592 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dd247c-2a31-452e-8b8e-00d146dc8aa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:43.115506 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-a8dd247c-2a31-452e-8b8e-00d146dc8aa0 None None] MFA Rules not processed for user `ba278ccf74424f729568a18219d92f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:26:43.116707 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-a8dd247c-2a31-452e-8b8e-00d146dc8aa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:26:43.135290 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1056/2103] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jun 29 11:26:42 2022] POST /identity/v3/projects => generated 369 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 11:26:43.139582 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dca526-d5bd-44ff-8821-fbb64336923b admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:43.172850 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:26:43.198575 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1048/2104] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:26:43 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 11:26:43.203984 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:26:43.209120 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dca526-d5bd-44ff-8821-fbb64336923b admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:43.210822 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dd247c-2a31-452e-8b8e-00d146dc8aa0 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:43.219402 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-a8dd247c-2a31-452e-8b8e-00d146dc8aa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:26:43.220159 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1aa1af-cf25-4651-ad18-5d6dd0052d03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:43.220159 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1aa1af-cf25-4651-ad18-5d6dd0052d03 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:43.220159 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1aa1af-cf25-4651-ad18-5d6dd0052d03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:43.220159 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1aa1af-cf25-4651-ad18-5d6dd0052d03 admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:43.220473 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1aa1af-cf25-4651-ad18-5d6dd0052d03 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:43.220473 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1aa1af-cf25-4651-ad18-5d6dd0052d03 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:43.272433 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dd247c-2a31-452e-8b8e-00d146dc8aa0 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:43.323083 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69b8288-6196-464b-af09-5af0916fdcaa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff7931b75a924394b1af733febce0834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:43.323083 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69b8288-6196-464b-af09-5af0916fdcaa service cinder] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:43.323083 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69b8288-6196-464b-af09-5af0916fdcaa service cinder] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:43.323083 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69b8288-6196-464b-af09-5af0916fdcaa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:43.323465 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-b69b8288-6196-464b-af09-5af0916fdcaa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:26:43.325953 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1057/2105] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jun 29 11:26:43 2022] POST /identity/v3/users => generated 291 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 11:26:43.330314 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1aa1af-cf25-4651-ad18-5d6dd0052d03 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:43.341196 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1aa1af-cf25-4651-ad18-5d6dd0052d03 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:43.347974 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743f4303-8736-4679-a899-1028b5b20182 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:43.348698 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743f4303-8736-4679-a899-1028b5b20182 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:43.359995 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743f4303-8736-4679-a899-1028b5b20182 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:43.359995 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743f4303-8736-4679-a899-1028b5b20182 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:43.359995 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743f4303-8736-4679-a899-1028b5b20182 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:43.359995 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743f4303-8736-4679-a899-1028b5b20182 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:43.364780 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1058/2106] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:26:43 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 11:26:43.365891 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743f4303-8736-4679-a899-1028b5b20182 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:43.376143 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743f4303-8736-4679-a899-1028b5b20182 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:43.380834 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b2954a-becf-4157-b070-71d1287af53f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:43.381699 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b2954a-becf-4157-b070-71d1287af53f admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:43.385912 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b2954a-becf-4157-b070-71d1287af53f admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:43.385912 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b2954a-becf-4157-b070-71d1287af53f admin admin] PATH_INFO: `/v3/projects/9698c67b3b7c43d4999c836d0a50a583/users/fb29c6b2851d408d9063414e6b98ec67/roles/e95c2ce63e114cc19a9dda6ad21e794f` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:43.390912 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69b8288-6196-464b-af09-5af0916fdcaa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:43.392447 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69b8288-6196-464b-af09-5af0916fdcaa service cinder] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:43.397848 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b2954a-becf-4157-b070-71d1287af53f admin admin] RBAC: Authorizing `identity:create_grant(project_id=9698c67b3b7c43d4999c836d0a50a583, user_id=fb29c6b2851d408d9063414e6b98ec67, role_id=e95c2ce63e114cc19a9dda6ad21e794f)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:43.400511 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b2954a-becf-4157-b070-71d1287af53f admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:43.401343 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1049/2107] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:26:43 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 11:26:43.404408 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:26:43.432985 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:26:43.467783 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1059/2108] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jun 29 11:26:43 2022] PUT /identity/v3/projects/9698c67b3b7c43d4999c836d0a50a583/users/fb29c6b2851d408d9063414e6b98ec67/roles/e95c2ce63e114cc19a9dda6ad21e794f => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 11:26:43.472353 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69b8288-6196-464b-af09-5af0916fdcaa service cinder] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:43.486814 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69b8288-6196-464b-af09-5af0916fdcaa service cinder] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:43.552062 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fb6331-4362-441c-a35b-96d49846d668 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:43.552741 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fb6331-4362-441c-a35b-96d49846d668 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:43.553023 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fb6331-4362-441c-a35b-96d49846d668 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:43.553327 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fb6331-4362-441c-a35b-96d49846d668 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:43.554063 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fb6331-4362-441c-a35b-96d49846d668 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:43.555760 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fb6331-4362-441c-a35b-96d49846d668 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:43.567585 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1050/2109] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:26:43 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 11:26:43.571975 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b2954a-becf-4157-b070-71d1287af53f admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:43.582970 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b2954a-becf-4157-b070-71d1287af53f admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:43.587586 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33d59e2-d50d-4dea-9103-191d6dfd69cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:43.588497 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33d59e2-d50d-4dea-9103-191d6dfd69cf admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:43.588786 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33d59e2-d50d-4dea-9103-191d6dfd69cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:43.589063 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33d59e2-d50d-4dea-9103-191d6dfd69cf admin admin] PATH_INFO: `/v3/projects/9698c67b3b7c43d4999c836d0a50a583/users/fb29c6b2851d408d9063414e6b98ec67/roles/e95c2ce63e114cc19a9dda6ad21e794f` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:43.591069 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33d59e2-d50d-4dea-9103-191d6dfd69cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=9698c67b3b7c43d4999c836d0a50a583, user_id=fb29c6b2851d408d9063414e6b98ec67, role_id=e95c2ce63e114cc19a9dda6ad21e794f)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:43.593729 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33d59e2-d50d-4dea-9103-191d6dfd69cf admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:43.604055 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1060/2110] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jun 29 11:26:43 2022] PUT /identity/v3/projects/9698c67b3b7c43d4999c836d0a50a583/users/fb29c6b2851d408d9063414e6b98ec67/roles/e95c2ce63e114cc19a9dda6ad21e794f => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:26:43.615093 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7129116c-6a60-40ed-873a-10c29dabd3a1 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:43.615383 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7129116c-6a60-40ed-873a-10c29dabd3a1 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:43.615660 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7129116c-6a60-40ed-873a-10c29dabd3a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:43.641660 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:26:43.644497 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-7129116c-6a60-40ed-873a-10c29dabd3a1 None None] MFA Rules not processed for user `fb29c6b2851d408d9063414e6b98ec67`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:26:43.646397 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-7129116c-6a60-40ed-873a-10c29dabd3a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:26:43.713102 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1051/2111] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:26:43 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:26:43.724703 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33d59e2-d50d-4dea-9103-191d6dfd69cf admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:43.726439 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-c33d59e2-d50d-4dea-9103-191d6dfd69cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:26:43.784404 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33d59e2-d50d-4dea-9103-191d6dfd69cf admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:43.788083 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df26ec35-4489-4fa6-bfd3-13fe618053df service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff7931b75a924394b1af733febce0834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:43.788734 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df26ec35-4489-4fa6-bfd3-13fe618053df service cinder] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:43.789021 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df26ec35-4489-4fa6-bfd3-13fe618053df service cinder] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:43.789298 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df26ec35-4489-4fa6-bfd3-13fe618053df service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:43.798704 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df26ec35-4489-4fa6-bfd3-13fe618053df service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:43.800162 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df26ec35-4489-4fa6-bfd3-13fe618053df service cinder] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:43.809746 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1061/2112] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:26:43 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:26:43.971066 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7129116c-6a60-40ed-873a-10c29dabd3a1 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:43.980599 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7129116c-6a60-40ed-873a-10c29dabd3a1 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:43.985446 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f212e38-b117-4b59-9c28-410ab3dc9835 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff7931b75a924394b1af733febce0834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:43.986102 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f212e38-b117-4b59-9c28-410ab3dc9835 service cinder] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:43.986389 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f212e38-b117-4b59-9c28-410ab3dc9835 service cinder] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:43.986668 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f212e38-b117-4b59-9c28-410ab3dc9835 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:44.046310 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f212e38-b117-4b59-9c28-410ab3dc9835 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:44.047745 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f212e38-b117-4b59-9c28-410ab3dc9835 service cinder] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:44.056986 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1052/2113] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:26:43 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:26:47.536608 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df26ec35-4489-4fa6-bfd3-13fe618053df service cinder] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:47.547272 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df26ec35-4489-4fa6-bfd3-13fe618053df service cinder] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:47.551166 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390383fe-b5ea-41c5-97a9-581b4ae4e586 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff7931b75a924394b1af733febce0834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:47.552100 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390383fe-b5ea-41c5-97a9-581b4ae4e586 service cinder] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:47.552389 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390383fe-b5ea-41c5-97a9-581b4ae4e586 service cinder] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:47.552721 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390383fe-b5ea-41c5-97a9-581b4ae4e586 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:47.610762 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390383fe-b5ea-41c5-97a9-581b4ae4e586 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:47.612250 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390383fe-b5ea-41c5-97a9-581b4ae4e586 service cinder] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:47.621481 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1062/2114] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:26:47 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:26:49.392396 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f212e38-b117-4b59-9c28-410ab3dc9835 service cinder] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:49.403224 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f212e38-b117-4b59-9c28-410ab3dc9835 service cinder] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:49.409361 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106bd673-6ff4-4722-8266-48295d9cfb17 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff7931b75a924394b1af733febce0834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:49.410118 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106bd673-6ff4-4722-8266-48295d9cfb17 service cinder] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:49.410609 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106bd673-6ff4-4722-8266-48295d9cfb17 service cinder] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:49.411082 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106bd673-6ff4-4722-8266-48295d9cfb17 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:49.492736 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106bd673-6ff4-4722-8266-48295d9cfb17 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:49.494208 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106bd673-6ff4-4722-8266-48295d9cfb17 service cinder] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:49.511870 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1053/2115] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:26:49 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:26:50.036748 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390383fe-b5ea-41c5-97a9-581b4ae4e586 service cinder] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:50.058598 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:26:50.101891 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390383fe-b5ea-41c5-97a9-581b4ae4e586 service cinder] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:50.106496 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdeaa471-1bc4-4344-9668-762f3c851cd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:50.107415 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeaa471-1bc4-4344-9668-762f3c851cd5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:50.111406 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeaa471-1bc4-4344-9668-762f3c851cd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:50.111406 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeaa471-1bc4-4344-9668-762f3c851cd5 admin admin] PATH_INFO: `/v3/users/594ddbf92764482eac3b5dc20464b3f4` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:50.111406 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdeaa471-1bc4-4344-9668-762f3c851cd5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=594ddbf92764482eac3b5dc20464b3f4)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:50.111406 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdeaa471-1bc4-4344-9668-762f3c851cd5 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:50.305666 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-bdeaa471-1bc4-4344-9668-762f3c851cd5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '594ddbf92764482eac3b5dc20464b3f4'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:50.306316 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-bdeaa471-1bc4-4344-9668-762f3c851cd5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '594ddbf92764482eac3b5dc20464b3f4'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:50.423918 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-bdeaa471-1bc4-4344-9668-762f3c851cd5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '594ddbf92764482eac3b5dc20464b3f4'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:50.433093 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-bdeaa471-1bc4-4344-9668-762f3c851cd5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '594ddbf92764482eac3b5dc20464b3f4'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:50.433785 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-bdeaa471-1bc4-4344-9668-762f3c851cd5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '594ddbf92764482eac3b5dc20464b3f4'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:50.470399 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1063/2116] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jun 29 11:26:50 2022] DELETE /identity/v3/users/594ddbf92764482eac3b5dc20464b3f4 => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:26:50.471813 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e5a5c5-1e34-47ba-a18e-c748c67a2677 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:50.472289 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e5a5c5-1e34-47ba-a18e-c748c67a2677 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:50.472398 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e5a5c5-1e34-47ba-a18e-c748c67a2677 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:50.489919 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:26:50.498506 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-a3e5a5c5-1e34-47ba-a18e-c748c67a2677 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:26:50.500057 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-a3e5a5c5-1e34-47ba-a18e-c748c67a2677 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:26:50.576132 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1054/2117] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:26:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:26:50.593589 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdeaa471-1bc4-4344-9668-762f3c851cd5 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:50.595653 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-bdeaa471-1bc4-4344-9668-762f3c851cd5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:26:50.696755 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdeaa471-1bc4-4344-9668-762f3c851cd5 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:50.702235 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b4708b-8039-4ad6-9cbe-fd0890bcf0e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f4bb9f989e4024a5da28dced646b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:50.703194 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b4708b-8039-4ad6-9cbe-fd0890bcf0e9 service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:50.703533 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b4708b-8039-4ad6-9cbe-fd0890bcf0e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:50.703990 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b4708b-8039-4ad6-9cbe-fd0890bcf0e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:50.715451 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b4708b-8039-4ad6-9cbe-fd0890bcf0e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:50.716947 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b4708b-8039-4ad6-9cbe-fd0890bcf0e9 service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:50.728766 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1064/2118] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:26:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:26:51.421817 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e5a5c5-1e34-47ba-a18e-c748c67a2677 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:51.422315 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-a3e5a5c5-1e34-47ba-a18e-c748c67a2677 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:26:51.445272 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e5a5c5-1e34-47ba-a18e-c748c67a2677 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:51.452298 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9163eb5-1949-4eef-961a-970b4513ed70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:51.453287 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9163eb5-1949-4eef-961a-970b4513ed70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:51.453667 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9163eb5-1949-4eef-961a-970b4513ed70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:51.454045 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9163eb5-1949-4eef-961a-970b4513ed70 admin admin] PATH_INFO: `/v3/projects/a5ad5c579bbd42a3ad80cbbd556f5b0b` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:51.455392 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9163eb5-1949-4eef-961a-970b4513ed70 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a5ad5c579bbd42a3ad80cbbd556f5b0b)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:51.457644 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9163eb5-1949-4eef-961a-970b4513ed70 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:51.687950 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-d9163eb5-1949-4eef-961a-970b4513ed70 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a5ad5c579bbd42a3ad80cbbd556f5b0b', 'request_id': 'req-d9163eb5-1949-4eef-961a-970b4513ed70'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:51.789751 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-d9163eb5-1949-4eef-961a-970b4513ed70 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a5ad5c579bbd42a3ad80cbbd556f5b0b', 'request_id': 'req-d9163eb5-1949-4eef-961a-970b4513ed70'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:51.796876 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-d9163eb5-1949-4eef-961a-970b4513ed70 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a5ad5c579bbd42a3ad80cbbd556f5b0b', 'request_id': 'req-d9163eb5-1949-4eef-961a-970b4513ed70'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:51.811532 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-d9163eb5-1949-4eef-961a-970b4513ed70 admin admin] The token cache is being invalidate because project a5ad5c579bbd42a3ad80cbbd556f5b0b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 11:26:51.812145 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-d9163eb5-1949-4eef-961a-970b4513ed70 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:51.813073 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-d9163eb5-1949-4eef-961a-970b4513ed70 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:51.814747 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1055/2119] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jun 29 11:26:51 2022] DELETE /identity/v3/projects/a5ad5c579bbd42a3ad80cbbd556f5b0b => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:26:51.819057 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b4708b-8039-4ad6-9cbe-fd0890bcf0e9 service neutron] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:51.835163 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b4708b-8039-4ad6-9cbe-fd0890bcf0e9 service neutron] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:51.899208 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:26:51.931693 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b3390c-58b6-4d11-b237-3035465981d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:51.933027 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b3390c-58b6-4d11-b237-3035465981d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:51.933373 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b3390c-58b6-4d11-b237-3035465981d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:51.933738 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b3390c-58b6-4d11-b237-3035465981d1 admin admin] PATH_INFO: `/v3/users/fb29c6b2851d408d9063414e6b98ec67` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:51.935737 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b3390c-58b6-4d11-b237-3035465981d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb29c6b2851d408d9063414e6b98ec67)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:51.937987 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b3390c-58b6-4d11-b237-3035465981d1 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:52.170115 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-41b3390c-58b6-4d11-b237-3035465981d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb29c6b2851d408d9063414e6b98ec67'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:52.170946 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-41b3390c-58b6-4d11-b237-3035465981d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb29c6b2851d408d9063414e6b98ec67'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:52.254076 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-41b3390c-58b6-4d11-b237-3035465981d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb29c6b2851d408d9063414e6b98ec67'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:52.273767 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-41b3390c-58b6-4d11-b237-3035465981d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb29c6b2851d408d9063414e6b98ec67'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:52.274835 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-41b3390c-58b6-4d11-b237-3035465981d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb29c6b2851d408d9063414e6b98ec67'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:52.300709 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1065/2120] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jun 29 11:26:51 2022] DELETE /identity/v3/users/fb29c6b2851d408d9063414e6b98ec67 => generated 0 bytes in 482 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 11:26:52.954080 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9163eb5-1949-4eef-961a-970b4513ed70 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:52.956498 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-d9163eb5-1949-4eef-961a-970b4513ed70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:26:52.973352 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9163eb5-1949-4eef-961a-970b4513ed70 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:53.061569 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9112f253-e57b-469d-ae33-e3566e86790b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:53.062565 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9112f253-e57b-469d-ae33-e3566e86790b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:53.062852 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9112f253-e57b-469d-ae33-e3566e86790b admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:53.063128 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9112f253-e57b-469d-ae33-e3566e86790b admin admin] PATH_INFO: `/v3/projects/9698c67b3b7c43d4999c836d0a50a583` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:53.064430 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9112f253-e57b-469d-ae33-e3566e86790b admin admin] RBAC: Authorizing `identity:delete_project(project_id=9698c67b3b7c43d4999c836d0a50a583)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:53.066495 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9112f253-e57b-469d-ae33-e3566e86790b admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:53.175774 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:26:53.188255 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-9112f253-e57b-469d-ae33-e3566e86790b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9698c67b3b7c43d4999c836d0a50a583', 'request_id': 'req-9112f253-e57b-469d-ae33-e3566e86790b'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:53.281172 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-9112f253-e57b-469d-ae33-e3566e86790b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9698c67b3b7c43d4999c836d0a50a583', 'request_id': 'req-9112f253-e57b-469d-ae33-e3566e86790b'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:53.288409 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-9112f253-e57b-469d-ae33-e3566e86790b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9698c67b3b7c43d4999c836d0a50a583', 'request_id': 'req-9112f253-e57b-469d-ae33-e3566e86790b'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:53.298947 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-9112f253-e57b-469d-ae33-e3566e86790b admin admin] The token cache is being invalidate because project 9698c67b3b7c43d4999c836d0a50a583 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 11:26:53.299235 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-9112f253-e57b-469d-ae33-e3566e86790b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:53.300093 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-9112f253-e57b-469d-ae33-e3566e86790b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:53.301803 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1056/2121] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jun 29 11:26:52 2022] DELETE /identity/v3/projects/9698c67b3b7c43d4999c836d0a50a583 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:26:53.307001 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2957a2-1d49-4a7a-85db-600a769fb164 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:53.307456 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2957a2-1d49-4a7a-85db-600a769fb164 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:53.307808 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2957a2-1d49-4a7a-85db-600a769fb164 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:53.319669 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e4abc9-0fe3-414b-9d7c-0ba78253f208 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:53.320086 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e4abc9-0fe3-414b-9d7c-0ba78253f208 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:53.320463 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e4abc9-0fe3-414b-9d7c-0ba78253f208 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:53.322650 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1057/2122] 10.222.0.44 () {54 vars in 921 bytes} [Wed Jun 29 11:26:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 29 11:26:53.327660 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-aa2957a2-1d49-4a7a-85db-600a769fb164 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:26:53.329075 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-aa2957a2-1d49-4a7a-85db-600a769fb164 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:26:53.338928 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5481452-db18-4dc3-b1c9-e32b472e9cec None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:53.339609 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5481452-db18-4dc3-b1c9-e32b472e9cec None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:53.340018 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5481452-db18-4dc3-b1c9-e32b472e9cec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:53.362797 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-d5481452-db18-4dc3-b1c9-e32b472e9cec None None] MFA Rules not processed for user `1a0505342b704d02977a5125957c2c9e`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:26:53.364056 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-d5481452-db18-4dc3-b1c9-e32b472e9cec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:26:53.419917 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1066/2123] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:26:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:26:53.425869 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e515fe-740a-41f4-8d48-e6e013968d97 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:53.426174 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e515fe-740a-41f4-8d48-e6e013968d97 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:53.426564 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e515fe-740a-41f4-8d48-e6e013968d97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:53.445297 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-77e515fe-740a-41f4-8d48-e6e013968d97 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:26:53.446561 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-77e515fe-740a-41f4-8d48-e6e013968d97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:26:53.453104 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1058/2124] 10.222.0.44 () {58 vars in 1018 bytes} [Wed Jun 29 11:26:53 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:26:53.454900 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1067/2125] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:26:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 11:26:53.460577 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5481452-db18-4dc3-b1c9-e32b472e9cec None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:53.464486 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e515fe-740a-41f4-8d48-e6e013968d97 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:53.466040 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-77e515fe-740a-41f4-8d48-e6e013968d97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:26:53.471397 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5481452-db18-4dc3-b1c9-e32b472e9cec None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:53.476225 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794e145e-1b6a-4f44-b8d9-235681dc8561 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:53.478497 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794e145e-1b6a-4f44-b8d9-235681dc8561 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:53.478497 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794e145e-1b6a-4f44-b8d9-235681dc8561 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:53.478497 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794e145e-1b6a-4f44-b8d9-235681dc8561 admin admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:53.478497 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794e145e-1b6a-4f44-b8d9-235681dc8561 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:26:53.478497 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794e145e-1b6a-4f44-b8d9-235681dc8561 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:53.480055 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794e145e-1b6a-4f44-b8d9-235681dc8561 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:53.480522 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e515fe-740a-41f4-8d48-e6e013968d97 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:53.486193 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b967ddb0-36dd-43e9-8e54-f030266982f8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a0505342b704d02977a5125957c2c9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:53.487015 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b967ddb0-36dd-43e9-8e54-f030266982f8 service placement] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:53.487324 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b967ddb0-36dd-43e9-8e54-f030266982f8 service placement] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:53.487628 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b967ddb0-36dd-43e9-8e54-f030266982f8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:53.493405 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1059/2126] 10.222.0.44 () {58 vars in 1152 bytes} [Wed Jun 29 11:26:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:26:53.499531 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794e145e-1b6a-4f44-b8d9-235681dc8561 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:53.502357 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b967ddb0-36dd-43e9-8e54-f030266982f8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:53.505356 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b967ddb0-36dd-43e9-8e54-f030266982f8 service placement] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:53.514578 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794e145e-1b6a-4f44-b8d9-235681dc8561 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:53.518166 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1068/2127] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:26:53 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 11:26:53.519194 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b967ddb0-36dd-43e9-8e54-f030266982f8 service placement] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:53.520546 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66582e43-9b76-4366-b943-6fac06371d4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:53.520700 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-b967ddb0-36dd-43e9-8e54-f030266982f8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:26:53.521235 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66582e43-9b76-4366-b943-6fac06371d4a admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:53.521521 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66582e43-9b76-4366-b943-6fac06371d4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:53.521797 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66582e43-9b76-4366-b943-6fac06371d4a admin admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:53.522495 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66582e43-9b76-4366-b943-6fac06371d4a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:53.524586 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66582e43-9b76-4366-b943-6fac06371d4a admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:53.542986 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b967ddb0-36dd-43e9-8e54-f030266982f8 service placement] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:53.549211 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361a9d48-bca5-4711-aeba-62e69795fa21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:53.552265 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361a9d48-bca5-4711-aeba-62e69795fa21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:53.552265 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361a9d48-bca5-4711-aeba-62e69795fa21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:53.552265 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361a9d48-bca5-4711-aeba-62e69795fa21 admin admin] PATH_INFO: `/v3/users/b1ca7ff32b434abc8d036ddfa266a584` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:53.559340 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361a9d48-bca5-4711-aeba-62e69795fa21 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b1ca7ff32b434abc8d036ddfa266a584)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:53.562022 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361a9d48-bca5-4711-aeba-62e69795fa21 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:53.622210 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1060/2128] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jun 29 11:26:53 2022] POST /identity/v3/projects => generated 367 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 11:26:53.626184 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66582e43-9b76-4366-b943-6fac06371d4a admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:53.687107 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66582e43-9b76-4366-b943-6fac06371d4a admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:53.691791 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8e279d-b3e4-40b4-a7d9-8d7d5cb1005a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:53.692491 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8e279d-b3e4-40b4-a7d9-8d7d5cb1005a admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:53.692794 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8e279d-b3e4-40b4-a7d9-8d7d5cb1005a admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:53.693238 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8e279d-b3e4-40b4-a7d9-8d7d5cb1005a admin admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:53.693999 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8e279d-b3e4-40b4-a7d9-8d7d5cb1005a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:53.695987 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8e279d-b3e4-40b4-a7d9-8d7d5cb1005a admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:53.784510 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:26:53.793121 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-361a9d48-bca5-4711-aeba-62e69795fa21 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b1ca7ff32b434abc8d036ddfa266a584'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:53.793809 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-361a9d48-bca5-4711-aeba-62e69795fa21 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b1ca7ff32b434abc8d036ddfa266a584'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:53.812587 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1061/2129] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jun 29 11:26:53 2022] POST /identity/v3/users => generated 284 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 11:26:53.812587 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8e279d-b3e4-40b4-a7d9-8d7d5cb1005a admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:53.813674 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-de8e279d-b3e4-40b4-a7d9-8d7d5cb1005a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:26:53.828378 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8e279d-b3e4-40b4-a7d9-8d7d5cb1005a admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:53.834792 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3ca4be-6723-4086-8a7a-97584850bc62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:53.835463 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3ca4be-6723-4086-8a7a-97584850bc62 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:53.835756 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3ca4be-6723-4086-8a7a-97584850bc62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:53.836071 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3ca4be-6723-4086-8a7a-97584850bc62 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:53.836813 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3ca4be-6723-4086-8a7a-97584850bc62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:53.838413 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3ca4be-6723-4086-8a7a-97584850bc62 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:53.844495 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:26:53.852963 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1062/2130] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:26:53 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 11:26:53.854133 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3ca4be-6723-4086-8a7a-97584850bc62 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:53.868058 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3ca4be-6723-4086-8a7a-97584850bc62 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:53.875140 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-361a9d48-bca5-4711-aeba-62e69795fa21 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b1ca7ff32b434abc8d036ddfa266a584'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:53.875994 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b1d492-9be9-4faa-94c5-2af1193677cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:53.876997 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b1d492-9be9-4faa-94c5-2af1193677cb admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:53.879755 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b1d492-9be9-4faa-94c5-2af1193677cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:53.879755 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b1d492-9be9-4faa-94c5-2af1193677cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:53.879755 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b1d492-9be9-4faa-94c5-2af1193677cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:53.881062 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b1d492-9be9-4faa-94c5-2af1193677cb admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:53.883567 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-361a9d48-bca5-4711-aeba-62e69795fa21 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b1ca7ff32b434abc8d036ddfa266a584'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:53.884340 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-361a9d48-bca5-4711-aeba-62e69795fa21 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b1ca7ff32b434abc8d036ddfa266a584'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:53.892777 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1063/2131] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:26:53 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 11:26:53.900545 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b1d492-9be9-4faa-94c5-2af1193677cb admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:53.909651 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1069/2132] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jun 29 11:26:53 2022] DELETE /identity/v3/users/b1ca7ff32b434abc8d036ddfa266a584 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:26:53.926618 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b1d492-9be9-4faa-94c5-2af1193677cb admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:54.008812 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361a9d48-bca5-4711-aeba-62e69795fa21 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:54.010480 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-361a9d48-bca5-4711-aeba-62e69795fa21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:26:54.014163 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273e6a84-a0af-43e9-b2fb-c0976e3aae3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:54.017813 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273e6a84-a0af-43e9-b2fb-c0976e3aae3a admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:54.017813 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273e6a84-a0af-43e9-b2fb-c0976e3aae3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:54.017813 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273e6a84-a0af-43e9-b2fb-c0976e3aae3a admin admin] PATH_INFO: `/v3/projects/c60c0b7eceb549df90814f699bd7f129/users/c2d7bd6b628949b085be11d67154a02e/roles/e1c88eebe911432cab6a07fbeb84653b` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:54.036250 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361a9d48-bca5-4711-aeba-62e69795fa21 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:54.039659 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273e6a84-a0af-43e9-b2fb-c0976e3aae3a admin admin] RBAC: Authorizing `identity:create_grant(project_id=c60c0b7eceb549df90814f699bd7f129, user_id=c2d7bd6b628949b085be11d67154a02e, role_id=e1c88eebe911432cab6a07fbeb84653b)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:54.043824 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273e6a84-a0af-43e9-b2fb-c0976e3aae3a admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:54.049775 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5d5221-c7c7-49c7-b62f-74d948954b96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:54.051644 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5d5221-c7c7-49c7-b62f-74d948954b96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:54.051968 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5d5221-c7c7-49c7-b62f-74d948954b96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:54.052291 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5d5221-c7c7-49c7-b62f-74d948954b96 admin admin] PATH_INFO: `/v3/users/c4ab39dfbe6d421b9aa35767d4f0d795` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:54.054537 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5d5221-c7c7-49c7-b62f-74d948954b96 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c4ab39dfbe6d421b9aa35767d4f0d795)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:54.057211 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5d5221-c7c7-49c7-b62f-74d948954b96 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:54.133854 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1064/2133] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jun 29 11:26:53 2022] PUT /identity/v3/projects/c60c0b7eceb549df90814f699bd7f129/users/c2d7bd6b628949b085be11d67154a02e/roles/e1c88eebe911432cab6a07fbeb84653b => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 11:26:54.148662 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9029af-8ac3-4cc8-b4d8-2c2d71b6df5d None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:54.148961 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9029af-8ac3-4cc8-b4d8-2c2d71b6df5d None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:54.149252 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9029af-8ac3-4cc8-b4d8-2c2d71b6df5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:54.191269 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-7a9029af-8ac3-4cc8-b4d8-2c2d71b6df5d None None] MFA Rules not processed for user `c2d7bd6b628949b085be11d67154a02e`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:26:54.196525 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-7a9029af-8ac3-4cc8-b4d8-2c2d71b6df5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:26:54.267359 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-9d5d5221-c7c7-49c7-b62f-74d948954b96 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c4ab39dfbe6d421b9aa35767d4f0d795'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:54.268054 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-9d5d5221-c7c7-49c7-b62f-74d948954b96 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c4ab39dfbe6d421b9aa35767d4f0d795'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:54.274682 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1065/2134] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:26:54 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:26:54.275800 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9029af-8ac3-4cc8-b4d8-2c2d71b6df5d None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:54.277227 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-7a9029af-8ac3-4cc8-b4d8-2c2d71b6df5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:26:54.361840 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-9d5d5221-c7c7-49c7-b62f-74d948954b96 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c4ab39dfbe6d421b9aa35767d4f0d795'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:54.366037 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9029af-8ac3-4cc8-b4d8-2c2d71b6df5d None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:54.370843 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db5f78b-b3f6-4e59-a90a-bb379a4b275f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:54.371703 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db5f78b-b3f6-4e59-a90a-bb379a4b275f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:54.371862 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-9d5d5221-c7c7-49c7-b62f-74d948954b96 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c4ab39dfbe6d421b9aa35767d4f0d795'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:54.372018 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db5f78b-b3f6-4e59-a90a-bb379a4b275f admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:54.372301 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db5f78b-b3f6-4e59-a90a-bb379a4b275f admin admin] PATH_INFO: `/v3/projects/07bc0c697e63498799ecf2a0335eae46` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:54.373648 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-9d5d5221-c7c7-49c7-b62f-74d948954b96 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c4ab39dfbe6d421b9aa35767d4f0d795'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:54.374122 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db5f78b-b3f6-4e59-a90a-bb379a4b275f admin admin] RBAC: Authorizing `identity:delete_project(project_id=07bc0c697e63498799ecf2a0335eae46)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:54.376093 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db5f78b-b3f6-4e59-a90a-bb379a4b275f admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:54.392180 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1070/2135] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jun 29 11:26:54 2022] DELETE /identity/v3/users/c4ab39dfbe6d421b9aa35767d4f0d795 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:26:54.395161 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5d5221-c7c7-49c7-b62f-74d948954b96 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:54.396043 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-9d5d5221-c7c7-49c7-b62f-74d948954b96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:26:54.461338 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5d5221-c7c7-49c7-b62f-74d948954b96 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:54.470797 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ed8ba9-9cd8-48a9-83cf-1b2e2dd8ccca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:54.472886 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ed8ba9-9cd8-48a9-83cf-1b2e2dd8ccca admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:54.472886 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ed8ba9-9cd8-48a9-83cf-1b2e2dd8ccca admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:54.472886 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ed8ba9-9cd8-48a9-83cf-1b2e2dd8ccca admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:54.472886 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ed8ba9-9cd8-48a9-83cf-1b2e2dd8ccca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:54.474904 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ed8ba9-9cd8-48a9-83cf-1b2e2dd8ccca admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:54.529431 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-4db5f78b-b3f6-4e59-a90a-bb379a4b275f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '07bc0c697e63498799ecf2a0335eae46', 'request_id': 'req-4db5f78b-b3f6-4e59-a90a-bb379a4b275f'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:54.627000 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1071/2136] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jun 29 11:26:54 2022] POST /identity/v3/projects => generated 365 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:26:54.628478 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02710726-9ed8-4604-ada3-a932365a96d2 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:54.628773 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02710726-9ed8-4604-ada3-a932365a96d2 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:54.629151 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02710726-9ed8-4604-ada3-a932365a96d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:54.647137 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-4db5f78b-b3f6-4e59-a90a-bb379a4b275f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '07bc0c697e63498799ecf2a0335eae46', 'request_id': 'req-4db5f78b-b3f6-4e59-a90a-bb379a4b275f'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:54.651481 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-02710726-9ed8-4604-ada3-a932365a96d2 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:26:54.653591 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-02710726-9ed8-4604-ada3-a932365a96d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:26:54.659760 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-4db5f78b-b3f6-4e59-a90a-bb379a4b275f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '07bc0c697e63498799ecf2a0335eae46', 'request_id': 'req-4db5f78b-b3f6-4e59-a90a-bb379a4b275f'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:54.661246 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=82,keystone:DELETE=15,keystone:INSERT=3 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:26:54.683524 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-4db5f78b-b3f6-4e59-a90a-bb379a4b275f admin admin] The token cache is being invalidate because project 07bc0c697e63498799ecf2a0335eae46 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 11:26:54.684337 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-4db5f78b-b3f6-4e59-a90a-bb379a4b275f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:54.685467 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-4db5f78b-b3f6-4e59-a90a-bb379a4b275f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:54.687488 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1066/2137] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jun 29 11:26:54 2022] DELETE /identity/v3/projects/07bc0c697e63498799ecf2a0335eae46 => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:26:54.688550 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db5f78b-b3f6-4e59-a90a-bb379a4b275f admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:54.694619 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-4db5f78b-b3f6-4e59-a90a-bb379a4b275f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:26:54.738650 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1072/2138] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:26:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:26:54.740079 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2d63d2-d5f5-4cc1-ab8c-d5ea99d339cf None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:54.740480 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2d63d2-d5f5-4cc1-ab8c-d5ea99d339cf None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:54.740859 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2d63d2-d5f5-4cc1-ab8c-d5ea99d339cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:54.761839 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-8d2d63d2-d5f5-4cc1-ab8c-d5ea99d339cf None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:26:54.763223 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-8d2d63d2-d5f5-4cc1-ab8c-d5ea99d339cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:26:54.771797 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1073/2139] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:26:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:26:54.777058 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2d63d2-d5f5-4cc1-ab8c-d5ea99d339cf None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:54.779441 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-8d2d63d2-d5f5-4cc1-ab8c-d5ea99d339cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:26:54.783822 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db5f78b-b3f6-4e59-a90a-bb379a4b275f admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:54.788815 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c666a5-6589-441a-bc8c-9c3375c2d401 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:54.789627 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c666a5-6589-441a-bc8c-9c3375c2d401 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:54.789971 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c666a5-6589-441a-bc8c-9c3375c2d401 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:54.790309 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c666a5-6589-441a-bc8c-9c3375c2d401 admin admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:54.791177 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c666a5-6589-441a-bc8c-9c3375c2d401 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:54.793181 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c666a5-6589-441a-bc8c-9c3375c2d401 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:54.850984 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2d63d2-d5f5-4cc1-ab8c-d5ea99d339cf None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:54.855514 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d55679b-ab40-4a68-ba91-a0b566f4133f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f4bb9f989e4024a5da28dced646b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:54.856359 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d55679b-ab40-4a68-ba91-a0b566f4133f service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:54.856810 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d55679b-ab40-4a68-ba91-a0b566f4133f service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:54.858004 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d55679b-ab40-4a68-ba91-a0b566f4133f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:54.873989 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d55679b-ab40-4a68-ba91-a0b566f4133f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:54.875494 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d55679b-ab40-4a68-ba91-a0b566f4133f service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:54.885707 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1074/2140] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:26:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:26:54.886985 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cc3027-99c6-475c-b951-5227cb616463 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:54.887277 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cc3027-99c6-475c-b951-5227cb616463 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:54.887609 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cc3027-99c6-475c-b951-5227cb616463 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:54.901407 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1067/2141] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jun 29 11:26:54 2022] POST /identity/v3/users => generated 289 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:26:54.906175 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c666a5-6589-441a-bc8c-9c3375c2d401 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:54.907599 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-e6cc3027-99c6-475c-b951-5227cb616463 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:26:54.909020 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-e6cc3027-99c6-475c-b951-5227cb616463 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:26:54.928485 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c666a5-6589-441a-bc8c-9c3375c2d401 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:54.931614 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1075/2142] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:26:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:26:54.932987 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b708921-da08-46e5-a083-7a8cd98ec57e None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:54.933270 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b708921-da08-46e5-a083-7a8cd98ec57e None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:54.933545 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b708921-da08-46e5-a083-7a8cd98ec57e None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:54.934996 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1076/2143] 10.222.0.44 () {54 vars in 921 bytes} [Wed Jun 29 11:26:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:26:54.936796 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b708921-da08-46e5-a083-7a8cd98ec57e None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:54.936923 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917deba5-e3a1-41be-975a-5147919b5678 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:54.937470 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917deba5-e3a1-41be-975a-5147919b5678 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:54.937755 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917deba5-e3a1-41be-975a-5147919b5678 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:54.938220 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917deba5-e3a1-41be-975a-5147919b5678 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:54.938967 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917deba5-e3a1-41be-975a-5147919b5678 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:54.941339 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917deba5-e3a1-41be-975a-5147919b5678 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:54.955711 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b708921-da08-46e5-a083-7a8cd98ec57e None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:54.963719 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1068/2144] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:26:54 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 11:26:54.963719 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfa3290-26b0-4725-80ed-187153116057 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:54.963719 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfa3290-26b0-4725-80ed-187153116057 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:54.963719 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfa3290-26b0-4725-80ed-187153116057 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:54.964077 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2723e24f-a81b-4fb5-ba4c-52b7bd70232b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:54.964077 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2723e24f-a81b-4fb5-ba4c-52b7bd70232b admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:54.964077 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2723e24f-a81b-4fb5-ba4c-52b7bd70232b admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:54.964325 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2723e24f-a81b-4fb5-ba4c-52b7bd70232b admin admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:54.965517 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2723e24f-a81b-4fb5-ba4c-52b7bd70232b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:26:54.965935 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2723e24f-a81b-4fb5-ba4c-52b7bd70232b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:54.968948 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2723e24f-a81b-4fb5-ba4c-52b7bd70232b admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:54.980210 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-0bfa3290-26b0-4725-80ed-187153116057 None None] MFA Rules not processed for user `1a0505342b704d02977a5125957c2c9e`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:26:54.981930 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-0bfa3290-26b0-4725-80ed-187153116057 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:26:54.987634 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1077/2145] 10.222.0.44 () {58 vars in 1152 bytes} [Wed Jun 29 11:26:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:26:54.990345 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2723e24f-a81b-4fb5-ba4c-52b7bd70232b admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:55.008675 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2723e24f-a81b-4fb5-ba4c-52b7bd70232b admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:55.025795 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848515f0-e085-4b1a-8f5c-2277f73856b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:55.026700 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848515f0-e085-4b1a-8f5c-2277f73856b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:55.026983 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848515f0-e085-4b1a-8f5c-2277f73856b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:55.027280 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848515f0-e085-4b1a-8f5c-2277f73856b1 admin admin] PATH_INFO: `/v3/projects/0ef7281e9e0e4133867869ed0c286bc1/users/df59f75672264d23bb649ac312afba47/roles/e95c2ce63e114cc19a9dda6ad21e794f` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:55.042889 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848515f0-e085-4b1a-8f5c-2277f73856b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ef7281e9e0e4133867869ed0c286bc1, user_id=df59f75672264d23bb649ac312afba47, role_id=e95c2ce63e114cc19a9dda6ad21e794f)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:55.047020 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848515f0-e085-4b1a-8f5c-2277f73856b1 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:55.056041 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1069/2146] 10.222.0.44 () {58 vars in 1018 bytes} [Wed Jun 29 11:26:54 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:26:55.056861 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfa3290-26b0-4725-80ed-187153116057 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:55.068744 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfa3290-26b0-4725-80ed-187153116057 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:55.076836 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4ba234-bb61-4317-ba60-18afe322b8b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:55.077512 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4ba234-bb61-4317-ba60-18afe322b8b3 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:55.077815 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4ba234-bb61-4317-ba60-18afe322b8b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:55.078090 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4ba234-bb61-4317-ba60-18afe322b8b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:55.078793 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4ba234-bb61-4317-ba60-18afe322b8b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:55.080762 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4ba234-bb61-4317-ba60-18afe322b8b3 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:55.119040 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:26:55.154317 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1078/2147] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jun 29 11:26:54 2022] PUT /identity/v3/projects/0ef7281e9e0e4133867869ed0c286bc1/users/df59f75672264d23bb649ac312afba47/roles/e95c2ce63e114cc19a9dda6ad21e794f => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:26:55.161288 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848515f0-e085-4b1a-8f5c-2277f73856b1 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:55.163669 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-848515f0-e085-4b1a-8f5c-2277f73856b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:26:55.233859 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1070/2148] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jun 29 11:26:55 2022] POST /identity/v3/projects => generated 387 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:26:55.235035 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4ba234-bb61-4317-ba60-18afe322b8b3 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:55.261773 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848515f0-e085-4b1a-8f5c-2277f73856b1 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:55.317327 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4ba234-bb61-4317-ba60-18afe322b8b3 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:55.324444 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eeca1d2-b953-4448-86e1-ef218b8c9030 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:55.325262 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eeca1d2-b953-4448-86e1-ef218b8c9030 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:55.326362 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eeca1d2-b953-4448-86e1-ef218b8c9030 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:55.326719 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eeca1d2-b953-4448-86e1-ef218b8c9030 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:55.327600 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eeca1d2-b953-4448-86e1-ef218b8c9030 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:55.329941 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eeca1d2-b953-4448-86e1-ef218b8c9030 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:55.334819 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3209fd4-bd02-49b4-a0d1-f9c2d3bf8bb3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a0505342b704d02977a5125957c2c9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:55.334819 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3209fd4-bd02-49b4-a0d1-f9c2d3bf8bb3 service placement] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:55.334819 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3209fd4-bd02-49b4-a0d1-f9c2d3bf8bb3 service placement] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:55.334819 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3209fd4-bd02-49b4-a0d1-f9c2d3bf8bb3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:55.343816 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3209fd4-bd02-49b4-a0d1-f9c2d3bf8bb3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:55.345896 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3209fd4-bd02-49b4-a0d1-f9c2d3bf8bb3 service placement] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:55.356718 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1079/2149] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:26:55 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:26:55.357570 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3209fd4-bd02-49b4-a0d1-f9c2d3bf8bb3 service placement] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:55.358361 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1071/2150] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:26:55 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 11:26:55.359243 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eeca1d2-b953-4448-86e1-ef218b8c9030 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:55.368705 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3209fd4-bd02-49b4-a0d1-f9c2d3bf8bb3 service placement] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:55.369371 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eeca1d2-b953-4448-86e1-ef218b8c9030 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:55.374891 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9114990-333b-4b07-bea0-806d4a1461dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:55.375302 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47b75d6-acc2-47ea-a5e5-c17c924f9d22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:55.375859 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9114990-333b-4b07-bea0-806d4a1461dc admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:55.376158 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9114990-333b-4b07-bea0-806d4a1461dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:55.376273 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47b75d6-acc2-47ea-a5e5-c17c924f9d22 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:55.376434 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9114990-333b-4b07-bea0-806d4a1461dc admin admin] PATH_INFO: `/v3/projects/0ef7281e9e0e4133867869ed0c286bc1/users/df59f75672264d23bb649ac312afba47/roles/e95c2ce63e114cc19a9dda6ad21e794f` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:55.376725 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47b75d6-acc2-47ea-a5e5-c17c924f9d22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:55.377092 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47b75d6-acc2-47ea-a5e5-c17c924f9d22 admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:55.378179 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47b75d6-acc2-47ea-a5e5-c17c924f9d22 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:55.378784 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9114990-333b-4b07-bea0-806d4a1461dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ef7281e9e0e4133867869ed0c286bc1, user_id=df59f75672264d23bb649ac312afba47, role_id=e95c2ce63e114cc19a9dda6ad21e794f)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:55.380465 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47b75d6-acc2-47ea-a5e5-c17c924f9d22 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:55.381254 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9114990-333b-4b07-bea0-806d4a1461dc admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:55.397361 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1072/2151] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jun 29 11:26:55 2022] PUT /identity/v3/projects/0ef7281e9e0e4133867869ed0c286bc1/users/df59f75672264d23bb649ac312afba47/roles/e95c2ce63e114cc19a9dda6ad21e794f => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:26:55.408797 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc38380-c6dc-4c1c-9815-459754112132 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:55.409129 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc38380-c6dc-4c1c-9815-459754112132 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:55.409434 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc38380-c6dc-4c1c-9815-459754112132 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:55.437124 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-cdc38380-c6dc-4c1c-9815-459754112132 None None] MFA Rules not processed for user `df59f75672264d23bb649ac312afba47`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:26:55.438356 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-cdc38380-c6dc-4c1c-9815-459754112132 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:26:55.525797 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1073/2152] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:26:55 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 11:26:55.526894 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc38380-c6dc-4c1c-9815-459754112132 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:55.528334 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-cdc38380-c6dc-4c1c-9815-459754112132 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:26:55.547340 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1080/2153] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jun 29 11:26:55 2022] POST /identity/v3/users => generated 294 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:26:55.548202 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47b75d6-acc2-47ea-a5e5-c17c924f9d22 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:55.554748 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-a47b75d6-acc2-47ea-a5e5-c17c924f9d22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:26:55.625142 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc38380-c6dc-4c1c-9815-459754112132 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:55.631013 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bade707-c0de-41a0-b0c3-4058d787d1d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:55.632644 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bade707-c0de-41a0-b0c3-4058d787d1d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:55.633189 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47b75d6-acc2-47ea-a5e5-c17c924f9d22 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:55.633511 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bade707-c0de-41a0-b0c3-4058d787d1d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:55.634526 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bade707-c0de-41a0-b0c3-4058d787d1d8 admin admin] PATH_INFO: `/v3/projects/a1c3a4798c2d4250abb6b87812a630e5` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:55.636513 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bade707-c0de-41a0-b0c3-4058d787d1d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a1c3a4798c2d4250abb6b87812a630e5)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:55.639794 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bade707-c0de-41a0-b0c3-4058d787d1d8 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:55.684830 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abdd506-7998-4f13-bc9d-b637e9baa89c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff7931b75a924394b1af733febce0834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:55.686093 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abdd506-7998-4f13-bc9d-b637e9baa89c service cinder] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:55.686391 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abdd506-7998-4f13-bc9d-b637e9baa89c service cinder] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:55.686658 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abdd506-7998-4f13-bc9d-b637e9baa89c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:55.696048 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abdd506-7998-4f13-bc9d-b637e9baa89c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:55.697570 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abdd506-7998-4f13-bc9d-b637e9baa89c service cinder] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:55.707033 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1081/2154] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:26:55 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:26:55.712191 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abdd506-7998-4f13-bc9d-b637e9baa89c service cinder] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:55.721943 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abdd506-7998-4f13-bc9d-b637e9baa89c service cinder] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:55.749717 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4d853d-206b-48df-9b9a-2ce3b667b1e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:55.750448 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4d853d-206b-48df-9b9a-2ce3b667b1e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:55.750737 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4d853d-206b-48df-9b9a-2ce3b667b1e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:55.751916 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4d853d-206b-48df-9b9a-2ce3b667b1e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:55.754436 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4d853d-206b-48df-9b9a-2ce3b667b1e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:55.757448 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4d853d-206b-48df-9b9a-2ce3b667b1e7 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:55.770181 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1082/2155] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:26:55 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 11:26:55.770181 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4d853d-206b-48df-9b9a-2ce3b667b1e7 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:55.772832 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-8b4d853d-206b-48df-9b9a-2ce3b667b1e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:26:55.777614 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-6bade707-c0de-41a0-b0c3-4058d787d1d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a1c3a4798c2d4250abb6b87812a630e5', 'request_id': 'req-6bade707-c0de-41a0-b0c3-4058d787d1d8'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:55.786768 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4d853d-206b-48df-9b9a-2ce3b667b1e7 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:55.850407 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-6bade707-c0de-41a0-b0c3-4058d787d1d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a1c3a4798c2d4250abb6b87812a630e5', 'request_id': 'req-6bade707-c0de-41a0-b0c3-4058d787d1d8'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:55.858186 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307c04d4-6d3a-448e-8c21-c745a0db39f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:55.859257 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307c04d4-6d3a-448e-8c21-c745a0db39f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:55.859655 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307c04d4-6d3a-448e-8c21-c745a0db39f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:55.860003 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307c04d4-6d3a-448e-8c21-c745a0db39f8 admin admin] PATH_INFO: `/v3/users/5ffed2ecca0c42fda7befa7a0934101c` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:55.862090 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-6bade707-c0de-41a0-b0c3-4058d787d1d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a1c3a4798c2d4250abb6b87812a630e5', 'request_id': 'req-6bade707-c0de-41a0-b0c3-4058d787d1d8'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:55.863255 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307c04d4-6d3a-448e-8c21-c745a0db39f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ffed2ecca0c42fda7befa7a0934101c)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:55.865669 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307c04d4-6d3a-448e-8c21-c745a0db39f8 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:55.898092 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-6bade707-c0de-41a0-b0c3-4058d787d1d8 admin admin] The token cache is being invalidate because project a1c3a4798c2d4250abb6b87812a630e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 11:26:55.898560 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-6bade707-c0de-41a0-b0c3-4058d787d1d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:55.899249 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-6bade707-c0de-41a0-b0c3-4058d787d1d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:55.901042 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1074/2156] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jun 29 11:26:55 2022] DELETE /identity/v3/projects/a1c3a4798c2d4250abb6b87812a630e5 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:26:55.901605 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bade707-c0de-41a0-b0c3-4058d787d1d8 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:55.902931 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-6bade707-c0de-41a0-b0c3-4058d787d1d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:26:55.921200 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bade707-c0de-41a0-b0c3-4058d787d1d8 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:55.926384 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a352c6-b304-4077-9c08-df13c3208925 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:55.927324 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a352c6-b304-4077-9c08-df13c3208925 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:55.927708 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a352c6-b304-4077-9c08-df13c3208925 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:55.928176 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a352c6-b304-4077-9c08-df13c3208925 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:55.929017 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a352c6-b304-4077-9c08-df13c3208925 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:55.930718 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a352c6-b304-4077-9c08-df13c3208925 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:55.948026 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1075/2157] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:26:55 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 11:26:55.949281 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a352c6-b304-4077-9c08-df13c3208925 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:55.951226 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-f9a352c6-b304-4077-9c08-df13c3208925 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:26:55.963718 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a352c6-b304-4077-9c08-df13c3208925 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:55.969115 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a7d364-3737-4327-a446-d689c78d952f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:55.970107 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a7d364-3737-4327-a446-d689c78d952f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:55.970404 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a7d364-3737-4327-a446-d689c78d952f admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:55.970741 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a7d364-3737-4327-a446-d689c78d952f admin admin] PATH_INFO: `/v3/users/331c576baffc429baec7d5058c8fdd1a` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:55.972678 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a7d364-3737-4327-a446-d689c78d952f admin admin] RBAC: Authorizing `identity:delete_user(user_id=331c576baffc429baec7d5058c8fdd1a)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:55.974829 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a7d364-3737-4327-a446-d689c78d952f admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:56.098408 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:26:56.112781 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-307c04d4-6d3a-448e-8c21-c745a0db39f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ffed2ecca0c42fda7befa7a0934101c'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:56.113482 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-307c04d4-6d3a-448e-8c21-c745a0db39f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ffed2ecca0c42fda7befa7a0934101c'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:56.168966 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-77a7d364-3737-4327-a446-d689c78d952f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '331c576baffc429baec7d5058c8fdd1a'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:56.179522 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-307c04d4-6d3a-448e-8c21-c745a0db39f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ffed2ecca0c42fda7befa7a0934101c'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:56.188535 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-307c04d4-6d3a-448e-8c21-c745a0db39f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ffed2ecca0c42fda7befa7a0934101c'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:56.189729 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-307c04d4-6d3a-448e-8c21-c745a0db39f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ffed2ecca0c42fda7befa7a0934101c'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:56.209352 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1083/2158] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jun 29 11:26:55 2022] DELETE /identity/v3/users/5ffed2ecca0c42fda7befa7a0934101c => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:26:56.210322 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307c04d4-6d3a-448e-8c21-c745a0db39f8 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:56.211775 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-307c04d4-6d3a-448e-8c21-c745a0db39f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:26:56.236042 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:26:56.252227 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-77a7d364-3737-4327-a446-d689c78d952f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '331c576baffc429baec7d5058c8fdd1a'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:56.252886 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-77a7d364-3737-4327-a446-d689c78d952f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '331c576baffc429baec7d5058c8fdd1a'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:56.265928 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-77a7d364-3737-4327-a446-d689c78d952f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '331c576baffc429baec7d5058c8fdd1a'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:56.266656 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-77a7d364-3737-4327-a446-d689c78d952f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '331c576baffc429baec7d5058c8fdd1a'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:56.279635 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1076/2159] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jun 29 11:26:55 2022] DELETE /identity/v3/users/331c576baffc429baec7d5058c8fdd1a => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:26:56.285570 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a7d364-3737-4327-a446-d689c78d952f admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:56.285795 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307c04d4-6d3a-448e-8c21-c745a0db39f8 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:56.285887 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-77a7d364-3737-4327-a446-d689c78d952f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:26:56.357996 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e36c68-9465-4cac-a509-b9e03856e552 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:56.358903 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e36c68-9465-4cac-a509-b9e03856e552 admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:56.359229 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e36c68-9465-4cac-a509-b9e03856e552 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:56.359517 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e36c68-9465-4cac-a509-b9e03856e552 admin admin] PATH_INFO: `/v3/projects/364a7eaf772c434e9e38a18430425895/users/275df17696084dce9e7af487275bacab/roles/e1c88eebe911432cab6a07fbeb84653b` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:56.373663 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a7d364-3737-4327-a446-d689c78d952f admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:56.374914 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e36c68-9465-4cac-a509-b9e03856e552 admin admin] RBAC: Authorizing `identity:create_grant(project_id=364a7eaf772c434e9e38a18430425895, user_id=275df17696084dce9e7af487275bacab, role_id=e1c88eebe911432cab6a07fbeb84653b)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:56.377861 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e36c68-9465-4cac-a509-b9e03856e552 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:56.379257 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af37507d-b071-45a5-aa14-8b31657518fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:56.380397 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af37507d-b071-45a5-aa14-8b31657518fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:56.381060 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af37507d-b071-45a5-aa14-8b31657518fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:56.381262 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af37507d-b071-45a5-aa14-8b31657518fa admin admin] PATH_INFO: `/v3/users/ccc29dd5c1eb45f08b0a18fb39fc93c4` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:56.382601 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af37507d-b071-45a5-aa14-8b31657518fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=ccc29dd5c1eb45f08b0a18fb39fc93c4)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:56.384635 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af37507d-b071-45a5-aa14-8b31657518fa admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:56.450269 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1084/2160] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jun 29 11:26:56 2022] PUT /identity/v3/projects/364a7eaf772c434e9e38a18430425895/users/275df17696084dce9e7af487275bacab/roles/e1c88eebe911432cab6a07fbeb84653b => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:26:56.452438 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e36c68-9465-4cac-a509-b9e03856e552 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:56.454422 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-19e36c68-9465-4cac-a509-b9e03856e552 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:26:56.514733 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e36c68-9465-4cac-a509-b9e03856e552 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:56.519679 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72e2378-9150-4278-b2a7-78214bbe5797 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff7931b75a924394b1af733febce0834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:56.520367 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72e2378-9150-4278-b2a7-78214bbe5797 service cinder] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:56.520653 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72e2378-9150-4278-b2a7-78214bbe5797 service cinder] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:56.520929 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72e2378-9150-4278-b2a7-78214bbe5797 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:56.522793 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-e72e2378-9150-4278-b2a7-78214bbe5797 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:26:56.578780 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-af37507d-b071-45a5-aa14-8b31657518fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ccc29dd5c1eb45f08b0a18fb39fc93c4'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:56.599756 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72e2378-9150-4278-b2a7-78214bbe5797 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:56.601223 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72e2378-9150-4278-b2a7-78214bbe5797 service cinder] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:56.610445 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1085/2161] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:26:56 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:26:56.611873 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f681b3d1-9542-40cc-bd85-28ef9e570bb9 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:56.612170 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f681b3d1-9542-40cc-bd85-28ef9e570bb9 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:56.612443 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f681b3d1-9542-40cc-bd85-28ef9e570bb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:56.685864 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-af37507d-b071-45a5-aa14-8b31657518fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ccc29dd5c1eb45f08b0a18fb39fc93c4'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:56.686540 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-af37507d-b071-45a5-aa14-8b31657518fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ccc29dd5c1eb45f08b0a18fb39fc93c4'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:56.701698 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-af37507d-b071-45a5-aa14-8b31657518fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ccc29dd5c1eb45f08b0a18fb39fc93c4'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:56.702489 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-af37507d-b071-45a5-aa14-8b31657518fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ccc29dd5c1eb45f08b0a18fb39fc93c4'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:56.715839 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-f681b3d1-9542-40cc-bd85-28ef9e570bb9 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:26:56.720695 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1077/2162] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jun 29 11:26:56 2022] DELETE /identity/v3/users/ccc29dd5c1eb45f08b0a18fb39fc93c4 => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:26:56.720924 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-f681b3d1-9542-40cc-bd85-28ef9e570bb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:26:56.721021 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb8c6c7-16fe-460f-bf26-ac4185d95490 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:56.721021 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb8c6c7-16fe-460f-bf26-ac4185d95490 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:56.721021 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb8c6c7-16fe-460f-bf26-ac4185d95490 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:56.742252 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-ecb8c6c7-16fe-460f-bf26-ac4185d95490 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:26:56.743944 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-ecb8c6c7-16fe-460f-bf26-ac4185d95490 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:26:56.819313 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1086/2163] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:26:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:26:56.819762 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1078/2164] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:26:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:26:56.820750 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb8c6c7-16fe-460f-bf26-ac4185d95490 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:56.822211 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-ecb8c6c7-16fe-460f-bf26-ac4185d95490 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:26:56.834455 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f681b3d1-9542-40cc-bd85-28ef9e570bb9 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:56.837750 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb8c6c7-16fe-460f-bf26-ac4185d95490 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:56.838637 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-f681b3d1-9542-40cc-bd85-28ef9e570bb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:26:56.846839 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b69bc4-5558-4668-9346-3881cc9782e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:56.848112 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b69bc4-5558-4668-9346-3881cc9782e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:56.848342 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b69bc4-5558-4668-9346-3881cc9782e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:56.848633 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b69bc4-5558-4668-9346-3881cc9782e4 admin admin] PATH_INFO: `/v3/projects/2ecbd73a067b46249fb587ced926d25a` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:56.850235 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b69bc4-5558-4668-9346-3881cc9782e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ecbd73a067b46249fb587ced926d25a)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:56.852207 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b69bc4-5558-4668-9346-3881cc9782e4 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:56.935746 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f681b3d1-9542-40cc-bd85-28ef9e570bb9 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:56.939539 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96c1e14-c620-4a9e-bf21-e187c8d29c4f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f4bb9f989e4024a5da28dced646b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:56.940227 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96c1e14-c620-4a9e-bf21-e187c8d29c4f service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:56.940516 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96c1e14-c620-4a9e-bf21-e187c8d29c4f service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:56.940793 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96c1e14-c620-4a9e-bf21-e187c8d29c4f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:56.950452 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96c1e14-c620-4a9e-bf21-e187c8d29c4f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:56.952481 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96c1e14-c620-4a9e-bf21-e187c8d29c4f service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:56.962638 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1087/2165] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:26:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:26:56.963375 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96c1e14-c620-4a9e-bf21-e187c8d29c4f service neutron] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:56.971593 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96c1e14-c620-4a9e-bf21-e187c8d29c4f service neutron] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:56.977899 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61199435-1232-4198-86dc-6072a19b6762 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f4bb9f989e4024a5da28dced646b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:56.978641 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61199435-1232-4198-86dc-6072a19b6762 service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:56.979047 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61199435-1232-4198-86dc-6072a19b6762 service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:56.979527 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61199435-1232-4198-86dc-6072a19b6762 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:56.989490 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61199435-1232-4198-86dc-6072a19b6762 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:56.991029 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61199435-1232-4198-86dc-6072a19b6762 service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:57.001542 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1088/2166] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:26:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:26:57.026503 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-a0b69bc4-5558-4668-9346-3881cc9782e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ecbd73a067b46249fb587ced926d25a', 'request_id': 'req-a0b69bc4-5558-4668-9346-3881cc9782e4'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:57.068901 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61199435-1232-4198-86dc-6072a19b6762 service neutron] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:57.070518 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-61199435-1232-4198-86dc-6072a19b6762 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:26:57.085058 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61199435-1232-4198-86dc-6072a19b6762 service neutron] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:57.118576 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-a0b69bc4-5558-4668-9346-3881cc9782e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ecbd73a067b46249fb587ced926d25a', 'request_id': 'req-a0b69bc4-5558-4668-9346-3881cc9782e4'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:57.128448 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-a0b69bc4-5558-4668-9346-3881cc9782e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ecbd73a067b46249fb587ced926d25a', 'request_id': 'req-a0b69bc4-5558-4668-9346-3881cc9782e4'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:57.149466 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-a0b69bc4-5558-4668-9346-3881cc9782e4 admin admin] The token cache is being invalidate because project 2ecbd73a067b46249fb587ced926d25a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 11:26:57.149861 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-a0b69bc4-5558-4668-9346-3881cc9782e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:57.150703 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-a0b69bc4-5558-4668-9346-3881cc9782e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:57.151386 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a3fd90-2407-442d-bf49-df9077dfea06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:57.152430 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a3fd90-2407-442d-bf49-df9077dfea06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:57.156757 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a3fd90-2407-442d-bf49-df9077dfea06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:57.156908 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1079/2167] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jun 29 11:26:56 2022] DELETE /identity/v3/projects/2ecbd73a067b46249fb587ced926d25a => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:26:57.157004 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a3fd90-2407-442d-bf49-df9077dfea06 admin admin] PATH_INFO: `/v3/projects/a16cac87d3c44485ab2bdff482d33052` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:57.157004 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a3fd90-2407-442d-bf49-df9077dfea06 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a16cac87d3c44485ab2bdff482d33052)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:57.157004 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a3fd90-2407-442d-bf49-df9077dfea06 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:57.158713 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedc32f1-26ec-4ee1-9fe9-56243ec56c3b None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:57.159267 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedc32f1-26ec-4ee1-9fe9-56243ec56c3b None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:57.159480 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedc32f1-26ec-4ee1-9fe9-56243ec56c3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:57.197155 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-eedc32f1-26ec-4ee1-9fe9-56243ec56c3b None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:26:57.198477 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-eedc32f1-26ec-4ee1-9fe9-56243ec56c3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:26:57.207629 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1080/2168] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:26:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:26:57.220455 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332b842f-170d-4c17-9836-753bd8249498 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:57.220748 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332b842f-170d-4c17-9836-753bd8249498 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:57.221329 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332b842f-170d-4c17-9836-753bd8249498 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:57.242008 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-332b842f-170d-4c17-9836-753bd8249498 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:26:57.243360 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-332b842f-170d-4c17-9836-753bd8249498 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:26:57.279001 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1081/2169] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:26:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 11:26:57.280006 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332b842f-170d-4c17-9836-753bd8249498 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:57.290034 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332b842f-170d-4c17-9836-753bd8249498 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:57.295888 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-66a3fd90-2407-442d-bf49-df9077dfea06 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a16cac87d3c44485ab2bdff482d33052', 'request_id': 'req-66a3fd90-2407-442d-bf49-df9077dfea06'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:57.315805 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-66a3fd90-2407-442d-bf49-df9077dfea06 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a16cac87d3c44485ab2bdff482d33052', 'request_id': 'req-66a3fd90-2407-442d-bf49-df9077dfea06'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:57.325559 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-66a3fd90-2407-442d-bf49-df9077dfea06 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a16cac87d3c44485ab2bdff482d33052', 'request_id': 'req-66a3fd90-2407-442d-bf49-df9077dfea06'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:57.367947 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eddb276-3206-40f4-9e22-2f1c2377b388 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:57.368891 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eddb276-3206-40f4-9e22-2f1c2377b388 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:57.369326 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eddb276-3206-40f4-9e22-2f1c2377b388 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:57.369679 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eddb276-3206-40f4-9e22-2f1c2377b388 admin admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:57.382454 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eddb276-3206-40f4-9e22-2f1c2377b388 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:26:57.383449 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eddb276-3206-40f4-9e22-2f1c2377b388 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:57.385135 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eddb276-3206-40f4-9e22-2f1c2377b388 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:57.403140 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1082/2170] 10.222.0.44 () {58 vars in 1152 bytes} [Wed Jun 29 11:26:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:26:57.409157 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eddb276-3206-40f4-9e22-2f1c2377b388 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:57.444899 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:26:57.481111 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-66a3fd90-2407-442d-bf49-df9077dfea06 admin admin] The token cache is being invalidate because project a16cac87d3c44485ab2bdff482d33052 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 11:26:57.481647 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-66a3fd90-2407-442d-bf49-df9077dfea06 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:57.483214 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-66a3fd90-2407-442d-bf49-df9077dfea06 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:57.488442 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1089/2171] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jun 29 11:26:57 2022] DELETE /identity/v3/projects/a16cac87d3c44485ab2bdff482d33052 => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 11:26:57.489484 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a3fd90-2407-442d-bf49-df9077dfea06 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:57.491332 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-66a3fd90-2407-442d-bf49-df9077dfea06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:26:57.503038 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eddb276-3206-40f4-9e22-2f1c2377b388 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:57.513010 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6e8548-a4b7-48ca-94a4-b87bc8f8ed18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:57.513973 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6e8548-a4b7-48ca-94a4-b87bc8f8ed18 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:57.514276 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6e8548-a4b7-48ca-94a4-b87bc8f8ed18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:57.514564 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6e8548-a4b7-48ca-94a4-b87bc8f8ed18 admin admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:57.515278 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6e8548-a4b7-48ca-94a4-b87bc8f8ed18 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:57.517491 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6e8548-a4b7-48ca-94a4-b87bc8f8ed18 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:57.559723 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a3fd90-2407-442d-bf49-df9077dfea06 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:57.565029 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d01faf-810c-434d-8c8f-a589382ac302 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:57.565549 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d01faf-810c-434d-8c8f-a589382ac302 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:57.570497 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d01faf-810c-434d-8c8f-a589382ac302 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:57.570497 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d01faf-810c-434d-8c8f-a589382ac302 admin admin] PATH_INFO: `/v3/users/27568bd41dba421f956eee06b3d3d0d3` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:57.570497 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d01faf-810c-434d-8c8f-a589382ac302 admin admin] RBAC: Authorizing `identity:delete_user(user_id=27568bd41dba421f956eee06b3d3d0d3)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:57.570497 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d01faf-810c-434d-8c8f-a589382ac302 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:57.664714 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1083/2172] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jun 29 11:26:57 2022] POST /identity/v3/projects => generated 385 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 11:26:57.665526 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6e8548-a4b7-48ca-94a4-b87bc8f8ed18 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:57.671132 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-0d6e8548-a4b7-48ca-94a4-b87bc8f8ed18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:26:57.734070 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6e8548-a4b7-48ca-94a4-b87bc8f8ed18 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:57.739185 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b771ff84-e1e1-4ed2-b2b6-16b6f0e902a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:57.740109 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b771ff84-e1e1-4ed2-b2b6-16b6f0e902a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:57.740399 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b771ff84-e1e1-4ed2-b2b6-16b6f0e902a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:57.740683 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b771ff84-e1e1-4ed2-b2b6-16b6f0e902a0 admin admin] PATH_INFO: `/v3/projects/3eefc925452241a0a4642d1ef46245a9` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:57.742048 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b771ff84-e1e1-4ed2-b2b6-16b6f0e902a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3eefc925452241a0a4642d1ef46245a9)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:57.747959 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b771ff84-e1e1-4ed2-b2b6-16b6f0e902a0 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:57.865769 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:26:57.878591 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-f9d01faf-810c-434d-8c8f-a589382ac302 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '27568bd41dba421f956eee06b3d3d0d3'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:57.879300 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-f9d01faf-810c-434d-8c8f-a589382ac302 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '27568bd41dba421f956eee06b3d3d0d3'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:57.939996 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-b771ff84-e1e1-4ed2-b2b6-16b6f0e902a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3eefc925452241a0a4642d1ef46245a9', 'request_id': 'req-b771ff84-e1e1-4ed2-b2b6-16b6f0e902a0'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:57.972661 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-f9d01faf-810c-434d-8c8f-a589382ac302 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '27568bd41dba421f956eee06b3d3d0d3'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:57.981354 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-f9d01faf-810c-434d-8c8f-a589382ac302 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '27568bd41dba421f956eee06b3d3d0d3'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:57.982132 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-f9d01faf-810c-434d-8c8f-a589382ac302 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '27568bd41dba421f956eee06b3d3d0d3'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:58.000011 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1090/2173] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jun 29 11:26:57 2022] DELETE /identity/v3/users/27568bd41dba421f956eee06b3d3d0d3 => generated 0 bytes in 511 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:26:58.001511 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d01faf-810c-434d-8c8f-a589382ac302 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:58.003319 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-f9d01faf-810c-434d-8c8f-a589382ac302 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:26:58.072526 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d01faf-810c-434d-8c8f-a589382ac302 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:58.078651 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-b771ff84-e1e1-4ed2-b2b6-16b6f0e902a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3eefc925452241a0a4642d1ef46245a9', 'request_id': 'req-b771ff84-e1e1-4ed2-b2b6-16b6f0e902a0'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:58.079008 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f68573e-8f72-40b0-9f4e-28dcf1c4cbc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:58.080448 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f68573e-8f72-40b0-9f4e-28dcf1c4cbc7 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:58.080807 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f68573e-8f72-40b0-9f4e-28dcf1c4cbc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:58.081081 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f68573e-8f72-40b0-9f4e-28dcf1c4cbc7 admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:58.081844 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f68573e-8f72-40b0-9f4e-28dcf1c4cbc7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:58.084200 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f68573e-8f72-40b0-9f4e-28dcf1c4cbc7 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:58.088587 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-b771ff84-e1e1-4ed2-b2b6-16b6f0e902a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3eefc925452241a0a4642d1ef46245a9', 'request_id': 'req-b771ff84-e1e1-4ed2-b2b6-16b6f0e902a0'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:58.102495 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-b771ff84-e1e1-4ed2-b2b6-16b6f0e902a0 admin admin] The token cache is being invalidate because project 3eefc925452241a0a4642d1ef46245a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 11:26:58.102895 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-b771ff84-e1e1-4ed2-b2b6-16b6f0e902a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:58.103643 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-b771ff84-e1e1-4ed2-b2b6-16b6f0e902a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:58.105715 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1084/2174] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jun 29 11:26:57 2022] DELETE /identity/v3/projects/3eefc925452241a0a4642d1ef46245a9 => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:26:58.109547 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b771ff84-e1e1-4ed2-b2b6-16b6f0e902a0 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:58.110937 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-b771ff84-e1e1-4ed2-b2b6-16b6f0e902a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:26:58.176057 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1091/2175] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jun 29 11:26:58 2022] POST /identity/v3/users => generated 293 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:26:58.180292 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f68573e-8f72-40b0-9f4e-28dcf1c4cbc7 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:58.181702 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b771ff84-e1e1-4ed2-b2b6-16b6f0e902a0 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:58.184957 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-5f68573e-8f72-40b0-9f4e-28dcf1c4cbc7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:26:58.188514 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4464d0-0cc3-4631-9be4-bc946cbf82b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:58.190852 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4464d0-0cc3-4631-9be4-bc946cbf82b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:58.191143 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4464d0-0cc3-4631-9be4-bc946cbf82b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:58.191423 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4464d0-0cc3-4631-9be4-bc946cbf82b6 admin admin] PATH_INFO: `/v3/users/ca65bb764a1841e6acc1f7feeea98a26` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:58.192741 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4464d0-0cc3-4631-9be4-bc946cbf82b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca65bb764a1841e6acc1f7feeea98a26)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:58.194767 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4464d0-0cc3-4631-9be4-bc946cbf82b6 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:58.198027 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f68573e-8f72-40b0-9f4e-28dcf1c4cbc7 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:58.203438 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0c6e35-e775-41fc-8676-54ecd652ae6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:58.204196 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0c6e35-e775-41fc-8676-54ecd652ae6c admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:58.204510 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0c6e35-e775-41fc-8676-54ecd652ae6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:58.204788 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0c6e35-e775-41fc-8676-54ecd652ae6c admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:58.205514 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0c6e35-e775-41fc-8676-54ecd652ae6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:58.207184 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0c6e35-e775-41fc-8676-54ecd652ae6c admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:58.218945 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1092/2176] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:26:58 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 11:26:58.223276 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0c6e35-e775-41fc-8676-54ecd652ae6c admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:58.233183 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0c6e35-e775-41fc-8676-54ecd652ae6c admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:58.237922 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1572cc-bd6d-4231-80ba-4ae45d0ab71c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:58.238575 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1572cc-bd6d-4231-80ba-4ae45d0ab71c admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:58.238863 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1572cc-bd6d-4231-80ba-4ae45d0ab71c admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:58.239142 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1572cc-bd6d-4231-80ba-4ae45d0ab71c admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:58.239922 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1572cc-bd6d-4231-80ba-4ae45d0ab71c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:58.241525 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1572cc-bd6d-4231-80ba-4ae45d0ab71c admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:58.253990 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1093/2177] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:26:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 11:26:58.257923 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1572cc-bd6d-4231-80ba-4ae45d0ab71c admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:58.267765 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1572cc-bd6d-4231-80ba-4ae45d0ab71c admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:58.282399 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9064ea6-1ba0-4086-a83c-9ce28ab28ebe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:58.283361 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9064ea6-1ba0-4086-a83c-9ce28ab28ebe admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:58.283649 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9064ea6-1ba0-4086-a83c-9ce28ab28ebe admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:58.284061 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9064ea6-1ba0-4086-a83c-9ce28ab28ebe admin admin] PATH_INFO: `/v3/projects/ce8bbb1b564245e1926ebf367203f2d7/users/8853be5b404e4554a06fdb9170552f9a/roles/e1c88eebe911432cab6a07fbeb84653b` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:58.311935 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9064ea6-1ba0-4086-a83c-9ce28ab28ebe admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce8bbb1b564245e1926ebf367203f2d7, user_id=8853be5b404e4554a06fdb9170552f9a, role_id=e1c88eebe911432cab6a07fbeb84653b)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:58.325109 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9064ea6-1ba0-4086-a83c-9ce28ab28ebe admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:58.366893 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-9f4464d0-0cc3-4631-9be4-bc946cbf82b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca65bb764a1841e6acc1f7feeea98a26'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:58.436084 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1094/2178] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jun 29 11:26:58 2022] PUT /identity/v3/projects/ce8bbb1b564245e1926ebf367203f2d7/users/8853be5b404e4554a06fdb9170552f9a/roles/e1c88eebe911432cab6a07fbeb84653b => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 11:26:58.436994 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9064ea6-1ba0-4086-a83c-9ce28ab28ebe admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:58.438775 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-f9064ea6-1ba0-4086-a83c-9ce28ab28ebe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:26:58.459119 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-9f4464d0-0cc3-4631-9be4-bc946cbf82b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca65bb764a1841e6acc1f7feeea98a26'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:58.461661 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-9f4464d0-0cc3-4631-9be4-bc946cbf82b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca65bb764a1841e6acc1f7feeea98a26'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:58.475193 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-9f4464d0-0cc3-4631-9be4-bc946cbf82b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca65bb764a1841e6acc1f7feeea98a26'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:58.476248 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-9f4464d0-0cc3-4631-9be4-bc946cbf82b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca65bb764a1841e6acc1f7feeea98a26'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:58.488898 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1085/2179] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jun 29 11:26:58 2022] DELETE /identity/v3/users/ca65bb764a1841e6acc1f7feeea98a26 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:26:58.490794 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce6fe48-80e5-4c52-ab02-ab5c9ec16fa6 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:58.490794 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce6fe48-80e5-4c52-ab02-ab5c9ec16fa6 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:58.490794 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce6fe48-80e5-4c52-ab02-ab5c9ec16fa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:58.511547 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9064ea6-1ba0-4086-a83c-9ce28ab28ebe admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:58.524036 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-2ce6fe48-80e5-4c52-ab02-ab5c9ec16fa6 None None] MFA Rules not processed for user `8853be5b404e4554a06fdb9170552f9a`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:26:58.525267 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-2ce6fe48-80e5-4c52-ab02-ab5c9ec16fa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:26:58.591097 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033ebe10-8642-4ddb-823c-30f68154a552 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:58.593506 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033ebe10-8642-4ddb-823c-30f68154a552 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:58.593506 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033ebe10-8642-4ddb-823c-30f68154a552 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:58.593506 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033ebe10-8642-4ddb-823c-30f68154a552 admin admin] PATH_INFO: `/v3/projects/bfd9203dfff74f1f864bfa6d10e34103` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:58.596391 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033ebe10-8642-4ddb-823c-30f68154a552 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bfd9203dfff74f1f864bfa6d10e34103)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:58.598705 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033ebe10-8642-4ddb-823c-30f68154a552 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:58.616374 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1086/2180] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:26:58 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:26:58.617097 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce6fe48-80e5-4c52-ab02-ab5c9ec16fa6 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:58.618537 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-2ce6fe48-80e5-4c52-ab02-ab5c9ec16fa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:26:58.633569 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce6fe48-80e5-4c52-ab02-ab5c9ec16fa6 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:58.638987 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9048e453-3eb1-4346-91d3-58b306894149 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:58.639680 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9048e453-3eb1-4346-91d3-58b306894149 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:58.639995 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9048e453-3eb1-4346-91d3-58b306894149 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:58.640274 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9048e453-3eb1-4346-91d3-58b306894149 admin admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:58.640998 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9048e453-3eb1-4346-91d3-58b306894149 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:58.642938 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9048e453-3eb1-4346-91d3-58b306894149 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:58.727985 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-033ebe10-8642-4ddb-823c-30f68154a552 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bfd9203dfff74f1f864bfa6d10e34103', 'request_id': 'req-033ebe10-8642-4ddb-823c-30f68154a552'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:58.736271 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-033ebe10-8642-4ddb-823c-30f68154a552 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bfd9203dfff74f1f864bfa6d10e34103', 'request_id': 'req-033ebe10-8642-4ddb-823c-30f68154a552'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:58.742384 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-033ebe10-8642-4ddb-823c-30f68154a552 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bfd9203dfff74f1f864bfa6d10e34103', 'request_id': 'req-033ebe10-8642-4ddb-823c-30f68154a552'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:58.764799 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1087/2181] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jun 29 11:26:58 2022] POST /identity/v3/projects => generated 387 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 11:26:58.767434 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9048e453-3eb1-4346-91d3-58b306894149 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:58.781648 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:26:58.832614 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-033ebe10-8642-4ddb-823c-30f68154a552 admin admin] The token cache is being invalidate because project bfd9203dfff74f1f864bfa6d10e34103 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 11:26:58.836159 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-033ebe10-8642-4ddb-823c-30f68154a552 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:58.836159 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-033ebe10-8642-4ddb-823c-30f68154a552 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:58.840167 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1095/2182] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jun 29 11:26:58 2022] DELETE /identity/v3/projects/bfd9203dfff74f1f864bfa6d10e34103 => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:26:58.842307 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4fd4c4-bf60-4651-9718-71a72fe112ee None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:58.842672 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4fd4c4-bf60-4651-9718-71a72fe112ee None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:58.843027 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4fd4c4-bf60-4651-9718-71a72fe112ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:58.847992 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9048e453-3eb1-4346-91d3-58b306894149 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:58.853137 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f2fa6e-65eb-4dbd-92f7-fc42f494a6c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:58.853813 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f2fa6e-65eb-4dbd-92f7-fc42f494a6c2 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:58.854271 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f2fa6e-65eb-4dbd-92f7-fc42f494a6c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:58.861417 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f2fa6e-65eb-4dbd-92f7-fc42f494a6c2 admin admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:58.862676 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f2fa6e-65eb-4dbd-92f7-fc42f494a6c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:58.864736 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f2fa6e-65eb-4dbd-92f7-fc42f494a6c2 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:58.901885 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-0c4fd4c4-bf60-4651-9718-71a72fe112ee None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:26:58.903272 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-0c4fd4c4-bf60-4651-9718-71a72fe112ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:26:58.918906 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1096/2183] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:26:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 11:26:58.926211 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994733e0-3d11-4e84-8a81-28e0aa1a159e None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:58.926920 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994733e0-3d11-4e84-8a81-28e0aa1a159e None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:58.927366 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994733e0-3d11-4e84-8a81-28e0aa1a159e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:58.951294 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-994733e0-3d11-4e84-8a81-28e0aa1a159e None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:26:58.952877 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-994733e0-3d11-4e84-8a81-28e0aa1a159e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:26:58.966298 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1097/2184] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:26:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 11:26:58.967405 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1088/2185] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jun 29 11:26:58 2022] POST /identity/v3/users => generated 300 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 11:26:58.967532 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994733e0-3d11-4e84-8a81-28e0aa1a159e None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:58.969180 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f2fa6e-65eb-4dbd-92f7-fc42f494a6c2 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:58.969512 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-994733e0-3d11-4e84-8a81-28e0aa1a159e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:26:58.983024 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f2fa6e-65eb-4dbd-92f7-fc42f494a6c2 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:58.988530 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63d7ab5-1931-4820-b41e-b2566dad903d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:58.989184 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63d7ab5-1931-4820-b41e-b2566dad903d admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:58.989587 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63d7ab5-1931-4820-b41e-b2566dad903d admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:58.989961 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63d7ab5-1931-4820-b41e-b2566dad903d admin admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:58.990774 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63d7ab5-1931-4820-b41e-b2566dad903d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:26:58.991271 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63d7ab5-1931-4820-b41e-b2566dad903d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:58.993367 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63d7ab5-1931-4820-b41e-b2566dad903d admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:58.995703 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994733e0-3d11-4e84-8a81-28e0aa1a159e None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:59.004183 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3b30a9-d389-460a-9fbc-c61ada4a8085 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:59.004883 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3b30a9-d389-460a-9fbc-c61ada4a8085 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:59.005160 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3b30a9-d389-460a-9fbc-c61ada4a8085 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:59.005437 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3b30a9-d389-460a-9fbc-c61ada4a8085 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:59.006172 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3b30a9-d389-460a-9fbc-c61ada4a8085 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:59.008308 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3b30a9-d389-460a-9fbc-c61ada4a8085 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:59.009747 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1089/2186] 10.222.0.44 () {58 vars in 1152 bytes} [Wed Jun 29 11:26:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:26:59.014216 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63d7ab5-1931-4820-b41e-b2566dad903d admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:59.027587 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1098/2187] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:26:58 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 11:26:59.029741 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3b30a9-d389-460a-9fbc-c61ada4a8085 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:59.031478 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63d7ab5-1931-4820-b41e-b2566dad903d admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:59.039205 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d21c135-db12-42a7-94a4-8ed0aae3051d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:59.040194 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d21c135-db12-42a7-94a4-8ed0aae3051d admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:59.040553 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d21c135-db12-42a7-94a4-8ed0aae3051d admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:59.040897 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d21c135-db12-42a7-94a4-8ed0aae3051d admin admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:59.042644 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d21c135-db12-42a7-94a4-8ed0aae3051d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:59.043271 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3b30a9-d389-460a-9fbc-c61ada4a8085 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:59.045495 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d21c135-db12-42a7-94a4-8ed0aae3051d admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:59.049183 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40055ecb-1bd6-4106-b88e-5c8f915976e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:59.050071 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40055ecb-1bd6-4106-b88e-5c8f915976e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:59.050494 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40055ecb-1bd6-4106-b88e-5c8f915976e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:59.050775 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40055ecb-1bd6-4106-b88e-5c8f915976e7 admin admin] PATH_INFO: `/v3/projects/57d1c6bf51b54b3eb1113a4b2d3e9850/users/95040e2e4cba4b5f958ffafdc29ec4ae/roles/e95c2ce63e114cc19a9dda6ad21e794f` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:59.067342 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40055ecb-1bd6-4106-b88e-5c8f915976e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=57d1c6bf51b54b3eb1113a4b2d3e9850, user_id=95040e2e4cba4b5f958ffafdc29ec4ae, role_id=e95c2ce63e114cc19a9dda6ad21e794f)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:59.070172 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40055ecb-1bd6-4106-b88e-5c8f915976e7 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:59.143817 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1090/2188] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jun 29 11:26:59 2022] POST /identity/v3/projects => generated 373 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:26:59.143817 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d21c135-db12-42a7-94a4-8ed0aae3051d admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:59.145598 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-1d21c135-db12-42a7-94a4-8ed0aae3051d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:26:59.158057 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1099/2189] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jun 29 11:26:59 2022] PUT /identity/v3/projects/57d1c6bf51b54b3eb1113a4b2d3e9850/users/95040e2e4cba4b5f958ffafdc29ec4ae/roles/e95c2ce63e114cc19a9dda6ad21e794f => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 11:26:59.159279 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40055ecb-1bd6-4106-b88e-5c8f915976e7 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:59.241049 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40055ecb-1bd6-4106-b88e-5c8f915976e7 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:59.246011 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72bf79c-247a-4847-9b4f-eb9063d15aeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:59.246856 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72bf79c-247a-4847-9b4f-eb9063d15aeb admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:59.247296 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72bf79c-247a-4847-9b4f-eb9063d15aeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:59.247581 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72bf79c-247a-4847-9b4f-eb9063d15aeb admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:59.248354 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72bf79c-247a-4847-9b4f-eb9063d15aeb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:59.250421 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72bf79c-247a-4847-9b4f-eb9063d15aeb admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:59.250968 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d21c135-db12-42a7-94a4-8ed0aae3051d admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:59.255666 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1f507f-42c2-4929-a4cc-e1c578248db0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:59.256558 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1f507f-42c2-4929-a4cc-e1c578248db0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:59.256848 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1f507f-42c2-4929-a4cc-e1c578248db0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:59.257134 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1f507f-42c2-4929-a4cc-e1c578248db0 admin admin] PATH_INFO: `/v3/projects/e2719678466341ab92f6ee2bee3ae3f2` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:59.258337 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1f507f-42c2-4929-a4cc-e1c578248db0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e2719678466341ab92f6ee2bee3ae3f2)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:59.260309 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1f507f-42c2-4929-a4cc-e1c578248db0 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:59.331884 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:26:59.339280 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1100/2190] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jun 29 11:26:59 2022] POST /identity/v3/users => generated 287 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:26:59.340377 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72bf79c-247a-4847-9b4f-eb9063d15aeb admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:59.351396 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72bf79c-247a-4847-9b4f-eb9063d15aeb admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:59.357425 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8a715b-0e1e-48cd-9d2c-cb51383d1ee1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:59.358329 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8a715b-0e1e-48cd-9d2c-cb51383d1ee1 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:59.358802 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8a715b-0e1e-48cd-9d2c-cb51383d1ee1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:59.359264 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8a715b-0e1e-48cd-9d2c-cb51383d1ee1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:59.360249 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8a715b-0e1e-48cd-9d2c-cb51383d1ee1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:59.362151 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8a715b-0e1e-48cd-9d2c-cb51383d1ee1 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:59.374157 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1101/2191] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:26:59 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 11:26:59.375163 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8a715b-0e1e-48cd-9d2c-cb51383d1ee1 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:59.389179 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8a715b-0e1e-48cd-9d2c-cb51383d1ee1 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:59.402639 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c470120-aaa3-4446-8b3e-ba93a0be378b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:59.403142 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c470120-aaa3-4446-8b3e-ba93a0be378b admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:59.403142 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c470120-aaa3-4446-8b3e-ba93a0be378b admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:59.403396 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c470120-aaa3-4446-8b3e-ba93a0be378b admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:59.404788 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c470120-aaa3-4446-8b3e-ba93a0be378b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:59.406404 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c470120-aaa3-4446-8b3e-ba93a0be378b admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:59.412756 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-7a1f507f-42c2-4929-a4cc-e1c578248db0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e2719678466341ab92f6ee2bee3ae3f2', 'request_id': 'req-7a1f507f-42c2-4929-a4cc-e1c578248db0'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:59.429679 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1102/2192] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:26:59 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 11:26:59.430765 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c470120-aaa3-4446-8b3e-ba93a0be378b admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:59.443131 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c470120-aaa3-4446-8b3e-ba93a0be378b admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:59.493342 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-7a1f507f-42c2-4929-a4cc-e1c578248db0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e2719678466341ab92f6ee2bee3ae3f2', 'request_id': 'req-7a1f507f-42c2-4929-a4cc-e1c578248db0'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:59.501595 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-7a1f507f-42c2-4929-a4cc-e1c578248db0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e2719678466341ab92f6ee2bee3ae3f2', 'request_id': 'req-7a1f507f-42c2-4929-a4cc-e1c578248db0'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:59.515870 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-7a1f507f-42c2-4929-a4cc-e1c578248db0 admin admin] The token cache is being invalidate because project e2719678466341ab92f6ee2bee3ae3f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 11:26:59.516216 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-7a1f507f-42c2-4929-a4cc-e1c578248db0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:59.517036 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-7a1f507f-42c2-4929-a4cc-e1c578248db0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:59.523973 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1091/2193] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jun 29 11:26:59 2022] DELETE /identity/v3/projects/e2719678466341ab92f6ee2bee3ae3f2 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:26:59.525740 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1f507f-42c2-4929-a4cc-e1c578248db0 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:59.529800 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-7a1f507f-42c2-4929-a4cc-e1c578248db0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:26:59.538220 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710b7055-8fb2-42d7-8fbe-0637afb946a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:59.539677 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710b7055-8fb2-42d7-8fbe-0637afb946a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:59.540209 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710b7055-8fb2-42d7-8fbe-0637afb946a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:59.540634 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710b7055-8fb2-42d7-8fbe-0637afb946a2 admin admin] PATH_INFO: `/v3/projects/57d1c6bf51b54b3eb1113a4b2d3e9850/users/95040e2e4cba4b5f958ffafdc29ec4ae/roles/e95c2ce63e114cc19a9dda6ad21e794f` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:59.542958 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710b7055-8fb2-42d7-8fbe-0637afb946a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=57d1c6bf51b54b3eb1113a4b2d3e9850, user_id=95040e2e4cba4b5f958ffafdc29ec4ae, role_id=e95c2ce63e114cc19a9dda6ad21e794f)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:59.544856 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1f507f-42c2-4929-a4cc-e1c578248db0 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:59.546575 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710b7055-8fb2-42d7-8fbe-0637afb946a2 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:59.550618 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd09936-c099-4fa5-99a3-1ded095e4e0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:59.551353 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd09936-c099-4fa5-99a3-1ded095e4e0f admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:59.551623 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd09936-c099-4fa5-99a3-1ded095e4e0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:59.551969 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd09936-c099-4fa5-99a3-1ded095e4e0f admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:59.552958 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd09936-c099-4fa5-99a3-1ded095e4e0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:59.554531 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd09936-c099-4fa5-99a3-1ded095e4e0f admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:59.562124 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1103/2194] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jun 29 11:26:59 2022] PUT /identity/v3/projects/57d1c6bf51b54b3eb1113a4b2d3e9850/users/95040e2e4cba4b5f958ffafdc29ec4ae/roles/e95c2ce63e114cc19a9dda6ad21e794f => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:26:59.563265 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710b7055-8fb2-42d7-8fbe-0637afb946a2 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:59.565476 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-710b7055-8fb2-42d7-8fbe-0637afb946a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:26:59.568936 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1092/2195] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:26:59 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 11:26:59.573168 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd09936-c099-4fa5-99a3-1ded095e4e0f admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:26:59.589103 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd09936-c099-4fa5-99a3-1ded095e4e0f admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:59.651557 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710b7055-8fb2-42d7-8fbe-0637afb946a2 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:26:59.658201 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4c8507-fb8f-4346-952c-e9d348e85a09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:59.659139 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4c8507-fb8f-4346-952c-e9d348e85a09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:59.659427 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4c8507-fb8f-4346-952c-e9d348e85a09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:59.659702 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4c8507-fb8f-4346-952c-e9d348e85a09 admin admin] PATH_INFO: `/v3/users/ba278ccf74424f729568a18219d92f8d` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:59.661373 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4c8507-fb8f-4346-952c-e9d348e85a09 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba278ccf74424f729568a18219d92f8d)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:59.669736 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af06aa4-8b22-4f05-b810-a4f7f8484fee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:26:59.670745 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af06aa4-8b22-4f05-b810-a4f7f8484fee admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:59.671319 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af06aa4-8b22-4f05-b810-a4f7f8484fee admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:59.671584 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af06aa4-8b22-4f05-b810-a4f7f8484fee admin admin] PATH_INFO: `/v3/projects/1b07dfb1ff6d4b51b1481435ecd8ca46/users/2e954d5b131f4ab1913a7c3c0b892b73/roles/e1c88eebe911432cab6a07fbeb84653b` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:59.676678 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4c8507-fb8f-4346-952c-e9d348e85a09 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:59.704555 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af06aa4-8b22-4f05-b810-a4f7f8484fee admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b07dfb1ff6d4b51b1481435ecd8ca46, user_id=2e954d5b131f4ab1913a7c3c0b892b73, role_id=e1c88eebe911432cab6a07fbeb84653b)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:26:59.707574 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af06aa4-8b22-4f05-b810-a4f7f8484fee admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:26:59.825103 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1093/2196] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jun 29 11:26:59 2022] PUT /identity/v3/projects/1b07dfb1ff6d4b51b1481435ecd8ca46/users/2e954d5b131f4ab1913a7c3c0b892b73/roles/e1c88eebe911432cab6a07fbeb84653b => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:26:59.827281 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fa4ef5-1f08-491a-a00d-849d8cb28c17 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:59.827811 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fa4ef5-1f08-491a-a00d-849d8cb28c17 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:59.828685 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fa4ef5-1f08-491a-a00d-849d8cb28c17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:26:59.859935 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-26fa4ef5-1f08-491a-a00d-849d8cb28c17 None None] MFA Rules not processed for user `95040e2e4cba4b5f958ffafdc29ec4ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:26:59.861674 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-26fa4ef5-1f08-491a-a00d-849d8cb28c17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:26:59.963436 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-4a4c8507-fb8f-4346-952c-e9d348e85a09 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba278ccf74424f729568a18219d92f8d'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:59.964231 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-4a4c8507-fb8f-4346-952c-e9d348e85a09 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba278ccf74424f729568a18219d92f8d'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:26:59.972779 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1094/2197] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:26:59 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:26:59.974496 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fe1990-5089-4fe9-8ffc-0c490ab19f79 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:26:59.974781 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fe1990-5089-4fe9-8ffc-0c490ab19f79 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:26:59.975058 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fe1990-5089-4fe9-8ffc-0c490ab19f79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:00.020219 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-27fe1990-5089-4fe9-8ffc-0c490ab19f79 None None] MFA Rules not processed for user `2e954d5b131f4ab1913a7c3c0b892b73`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:27:00.021874 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-27fe1990-5089-4fe9-8ffc-0c490ab19f79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:27:00.068536 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-4a4c8507-fb8f-4346-952c-e9d348e85a09 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba278ccf74424f729568a18219d92f8d'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:27:00.078682 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-4a4c8507-fb8f-4346-952c-e9d348e85a09 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba278ccf74424f729568a18219d92f8d'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:27:00.079615 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-4a4c8507-fb8f-4346-952c-e9d348e85a09 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba278ccf74424f729568a18219d92f8d'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:27:00.116584 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1095/2198] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:26:59 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:27:00.117409 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fe1990-5089-4fe9-8ffc-0c490ab19f79 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:27:00.118697 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-27fe1990-5089-4fe9-8ffc-0c490ab19f79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:27:00.134049 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1104/2199] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jun 29 11:26:59 2022] DELETE /identity/v3/users/ba278ccf74424f729568a18219d92f8d => generated 0 bytes in 571 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:27:00.220244 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fe1990-5089-4fe9-8ffc-0c490ab19f79 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:27:00.264983 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4c8507-fb8f-4346-952c-e9d348e85a09 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:27:00.266641 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-4a4c8507-fb8f-4346-952c-e9d348e85a09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:27:00.306355 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1b3eb6-434f-4693-a436-168923ba6514 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff7931b75a924394b1af733febce0834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:27:00.307019 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1b3eb6-434f-4693-a436-168923ba6514 service cinder] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:00.307355 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1b3eb6-434f-4693-a436-168923ba6514 service cinder] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:00.307570 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1b3eb6-434f-4693-a436-168923ba6514 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:00.310910 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-ab1b3eb6-434f-4693-a436-168923ba6514 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:27:00.348446 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4c8507-fb8f-4346-952c-e9d348e85a09 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:27:00.352581 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c6958c-b76d-4060-aef1-879a65a79da4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff7931b75a924394b1af733febce0834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:27:00.353247 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c6958c-b76d-4060-aef1-879a65a79da4 service cinder] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:00.353536 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c6958c-b76d-4060-aef1-879a65a79da4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:00.353812 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c6958c-b76d-4060-aef1-879a65a79da4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:00.398042 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1b3eb6-434f-4693-a436-168923ba6514 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:27:00.400472 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1b3eb6-434f-4693-a436-168923ba6514 service cinder] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:27:00.422695 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c6958c-b76d-4060-aef1-879a65a79da4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:27:00.424490 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c6958c-b76d-4060-aef1-879a65a79da4 service cinder] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:27:00.425052 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1096/2200] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:27:00 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:27:00.442302 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1105/2201] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:27:00 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 11:27:00.478454 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1b3eb6-434f-4693-a436-168923ba6514 service cinder] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:27:00.480165 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-ab1b3eb6-434f-4693-a436-168923ba6514 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:27:00.586696 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1b3eb6-434f-4693-a436-168923ba6514 service cinder] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:27:00.597438 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a73fa60-6d06-4ded-a3ae-83f1644d2749 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:27:00.597438 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a73fa60-6d06-4ded-a3ae-83f1644d2749 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:00.597438 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a73fa60-6d06-4ded-a3ae-83f1644d2749 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:00.597438 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a73fa60-6d06-4ded-a3ae-83f1644d2749 admin admin] PATH_INFO: `/v3/users/2e954d5b131f4ab1913a7c3c0b892b73` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:00.597724 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a73fa60-6d06-4ded-a3ae-83f1644d2749 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e954d5b131f4ab1913a7c3c0b892b73)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:27:00.599824 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a73fa60-6d06-4ded-a3ae-83f1644d2749 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:27:00.845695 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-8a73fa60-6d06-4ded-a3ae-83f1644d2749 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e954d5b131f4ab1913a7c3c0b892b73'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:27:00.968788 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:27:00.971859 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-8a73fa60-6d06-4ded-a3ae-83f1644d2749 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e954d5b131f4ab1913a7c3c0b892b73'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:27:00.973471 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-8a73fa60-6d06-4ded-a3ae-83f1644d2749 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e954d5b131f4ab1913a7c3c0b892b73'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:27:00.980342 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c6958c-b76d-4060-aef1-879a65a79da4 service cinder] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:27:00.981945 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-f0c6958c-b76d-4060-aef1-879a65a79da4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:27:00.998533 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c6958c-b76d-4060-aef1-879a65a79da4 service cinder] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:27:01.004169 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da7d8d8-01ae-4e99-9e4c-7edfa8baf61f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:27:01.005261 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da7d8d8-01ae-4e99-9e4c-7edfa8baf61f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:01.005956 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da7d8d8-01ae-4e99-9e4c-7edfa8baf61f admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:01.006629 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da7d8d8-01ae-4e99-9e4c-7edfa8baf61f admin admin] PATH_INFO: `/v3/projects/4657d8718c8f4ef2890100d9b33cf31e` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:01.008256 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da7d8d8-01ae-4e99-9e4c-7edfa8baf61f admin admin] RBAC: Authorizing `identity:delete_project(project_id=4657d8718c8f4ef2890100d9b33cf31e)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:27:01.010754 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da7d8d8-01ae-4e99-9e4c-7edfa8baf61f admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:27:01.023686 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-8a73fa60-6d06-4ded-a3ae-83f1644d2749 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e954d5b131f4ab1913a7c3c0b892b73'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:27:01.025453 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-8a73fa60-6d06-4ded-a3ae-83f1644d2749 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e954d5b131f4ab1913a7c3c0b892b73'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:27:01.061938 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1097/2202] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jun 29 11:27:00 2022] DELETE /identity/v3/users/2e954d5b131f4ab1913a7c3c0b892b73 => generated 0 bytes in 583 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:27:01.065180 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6376098f-3300-497d-b2a7-b17b35cf6b2e None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:01.065476 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6376098f-3300-497d-b2a7-b17b35cf6b2e None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:01.065761 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6376098f-3300-497d-b2a7-b17b35cf6b2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:01.102352 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-6376098f-3300-497d-b2a7-b17b35cf6b2e None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:27:01.103610 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-6376098f-3300-497d-b2a7-b17b35cf6b2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:27:01.173043 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-7da7d8d8-01ae-4e99-9e4c-7edfa8baf61f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4657d8718c8f4ef2890100d9b33cf31e', 'request_id': 'req-7da7d8d8-01ae-4e99-9e4c-7edfa8baf61f'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:27:01.178656 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-7da7d8d8-01ae-4e99-9e4c-7edfa8baf61f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4657d8718c8f4ef2890100d9b33cf31e', 'request_id': 'req-7da7d8d8-01ae-4e99-9e4c-7edfa8baf61f'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:27:01.183037 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1098/2203] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:27:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:27:01.186318 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-7da7d8d8-01ae-4e99-9e4c-7edfa8baf61f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4657d8718c8f4ef2890100d9b33cf31e', 'request_id': 'req-7da7d8d8-01ae-4e99-9e4c-7edfa8baf61f'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:27:01.193291 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6376098f-3300-497d-b2a7-b17b35cf6b2e None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:27:01.195086 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-6376098f-3300-497d-b2a7-b17b35cf6b2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:27:01.272894 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-7da7d8d8-01ae-4e99-9e4c-7edfa8baf61f admin admin] The token cache is being invalidate because project 4657d8718c8f4ef2890100d9b33cf31e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 11:27:01.273319 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-7da7d8d8-01ae-4e99-9e4c-7edfa8baf61f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:27:01.274332 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-7da7d8d8-01ae-4e99-9e4c-7edfa8baf61f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:27:01.276870 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1106/2204] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jun 29 11:27:00 2022] DELETE /identity/v3/projects/4657d8718c8f4ef2890100d9b33cf31e => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:27:01.327133 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6376098f-3300-497d-b2a7-b17b35cf6b2e None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:27:01.462604 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad61fa69-e878-4ed1-98d6-4797ccb8a150 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f4bb9f989e4024a5da28dced646b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:27:01.463369 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad61fa69-e878-4ed1-98d6-4797ccb8a150 service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:01.463755 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad61fa69-e878-4ed1-98d6-4797ccb8a150 service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:01.464135 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad61fa69-e878-4ed1-98d6-4797ccb8a150 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:01.466805 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-ad61fa69-e878-4ed1-98d6-4797ccb8a150 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:27:01.555457 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad61fa69-e878-4ed1-98d6-4797ccb8a150 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:27:01.557485 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad61fa69-e878-4ed1-98d6-4797ccb8a150 service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:27:01.567902 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1099/2205] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:27:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:27:02.065758 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:27:02.096574 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da7d8d8-01ae-4e99-9e4c-7edfa8baf61f admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:27:02.098206 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-7da7d8d8-01ae-4e99-9e4c-7edfa8baf61f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:27:02.110965 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad61fa69-e878-4ed1-98d6-4797ccb8a150 service neutron] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:27:02.112742 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-ad61fa69-e878-4ed1-98d6-4797ccb8a150 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:27:02.123692 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da7d8d8-01ae-4e99-9e4c-7edfa8baf61f admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:27:02.131548 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9839b6f-85a7-4df5-8fdf-d2c9e9945f64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:27:02.131975 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad61fa69-e878-4ed1-98d6-4797ccb8a150 service neutron] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:27:02.132896 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9839b6f-85a7-4df5-8fdf-d2c9e9945f64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:02.133346 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9839b6f-85a7-4df5-8fdf-d2c9e9945f64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:02.133788 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9839b6f-85a7-4df5-8fdf-d2c9e9945f64 admin admin] PATH_INFO: `/v3/users/8853be5b404e4554a06fdb9170552f9a` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:02.137407 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9839b6f-85a7-4df5-8fdf-d2c9e9945f64 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8853be5b404e4554a06fdb9170552f9a)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:27:02.139279 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9839b6f-85a7-4df5-8fdf-d2c9e9945f64 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:27:02.147033 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d900398b-f86c-4cfd-b14c-129fb1f53523 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:27:02.147335 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d900398b-f86c-4cfd-b14c-129fb1f53523 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:02.147651 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d900398b-f86c-4cfd-b14c-129fb1f53523 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:02.147938 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d900398b-f86c-4cfd-b14c-129fb1f53523 admin admin] PATH_INFO: `/v3/projects/1b07dfb1ff6d4b51b1481435ecd8ca46` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:02.149556 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d900398b-f86c-4cfd-b14c-129fb1f53523 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1b07dfb1ff6d4b51b1481435ecd8ca46)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:27:02.151481 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d900398b-f86c-4cfd-b14c-129fb1f53523 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:27:02.394238 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-a9839b6f-85a7-4df5-8fdf-d2c9e9945f64 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8853be5b404e4554a06fdb9170552f9a'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:27:02.394980 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-a9839b6f-85a7-4df5-8fdf-d2c9e9945f64 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8853be5b404e4554a06fdb9170552f9a'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:27:02.405082 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-d900398b-f86c-4cfd-b14c-129fb1f53523 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1b07dfb1ff6d4b51b1481435ecd8ca46', 'request_id': 'req-d900398b-f86c-4cfd-b14c-129fb1f53523'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:27:02.486850 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-a9839b6f-85a7-4df5-8fdf-d2c9e9945f64 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8853be5b404e4554a06fdb9170552f9a'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:27:02.495213 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-a9839b6f-85a7-4df5-8fdf-d2c9e9945f64 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8853be5b404e4554a06fdb9170552f9a'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:27:02.496790 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-a9839b6f-85a7-4df5-8fdf-d2c9e9945f64 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8853be5b404e4554a06fdb9170552f9a'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:27:02.523242 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1107/2206] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jun 29 11:27:02 2022] DELETE /identity/v3/users/8853be5b404e4554a06fdb9170552f9a => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:27:02.525105 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1c91cd-86a7-427b-ad3e-9afb0484e1a7 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:02.525585 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1c91cd-86a7-427b-ad3e-9afb0484e1a7 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:02.525921 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1c91cd-86a7-427b-ad3e-9afb0484e1a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:02.546540 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-5b1c91cd-86a7-427b-ad3e-9afb0484e1a7 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:27:02.548011 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-5b1c91cd-86a7-427b-ad3e-9afb0484e1a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:27:02.599813 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-d900398b-f86c-4cfd-b14c-129fb1f53523 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1b07dfb1ff6d4b51b1481435ecd8ca46', 'request_id': 'req-d900398b-f86c-4cfd-b14c-129fb1f53523'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:27:02.614799 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-d900398b-f86c-4cfd-b14c-129fb1f53523 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1b07dfb1ff6d4b51b1481435ecd8ca46', 'request_id': 'req-d900398b-f86c-4cfd-b14c-129fb1f53523'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:27:02.634206 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-d900398b-f86c-4cfd-b14c-129fb1f53523 admin admin] The token cache is being invalidate because project 1b07dfb1ff6d4b51b1481435ecd8ca46 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 11:27:02.634563 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-d900398b-f86c-4cfd-b14c-129fb1f53523 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:27:02.636237 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-d900398b-f86c-4cfd-b14c-129fb1f53523 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:27:02.638355 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1100/2207] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jun 29 11:27:02 2022] DELETE /identity/v3/projects/1b07dfb1ff6d4b51b1481435ecd8ca46 => generated 0 bytes in 528 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:27:02.645129 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf7c72b-e9d7-4ebb-83f3-28669dff4dcd None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:02.653137 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf7c72b-e9d7-4ebb-83f3-28669dff4dcd None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:02.654023 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1108/2208] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:27:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 11:27:02.654146 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf7c72b-e9d7-4ebb-83f3-28669dff4dcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:02.660404 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1c91cd-86a7-427b-ad3e-9afb0484e1a7 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:27:02.661974 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-5b1c91cd-86a7-427b-ad3e-9afb0484e1a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:27:02.682258 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-7bf7c72b-e9d7-4ebb-83f3-28669dff4dcd None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:27:02.683760 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-7bf7c72b-e9d7-4ebb-83f3-28669dff4dcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:27:02.694951 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1101/2209] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:27:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:27:02.703107 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af46c7f-3398-4f70-b0c7-9b27b5ddf03a None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:02.704154 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af46c7f-3398-4f70-b0c7-9b27b5ddf03a None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:02.704616 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af46c7f-3398-4f70-b0c7-9b27b5ddf03a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:02.742418 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-7af46c7f-3398-4f70-b0c7-9b27b5ddf03a None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:27:02.743895 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-7af46c7f-3398-4f70-b0c7-9b27b5ddf03a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:27:02.754272 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1102/2210] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:27:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:27:02.759333 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af46c7f-3398-4f70-b0c7-9b27b5ddf03a None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:27:02.782645 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af46c7f-3398-4f70-b0c7-9b27b5ddf03a None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:27:02.788581 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09c0372-e3bd-4283-96bf-fde2585a94ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:27:02.789823 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09c0372-e3bd-4283-96bf-fde2585a94ac admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:02.790396 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09c0372-e3bd-4283-96bf-fde2585a94ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:02.791028 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09c0372-e3bd-4283-96bf-fde2585a94ac admin admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:02.792081 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09c0372-e3bd-4283-96bf-fde2585a94ac admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:27:02.792404 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09c0372-e3bd-4283-96bf-fde2585a94ac admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:27:02.801849 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09c0372-e3bd-4283-96bf-fde2585a94ac admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:27:02.808281 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1c91cd-86a7-427b-ad3e-9afb0484e1a7 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:27:02.813413 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1103/2211] 10.222.0.44 () {58 vars in 1152 bytes} [Wed Jun 29 11:27:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:27:02.814370 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09c0372-e3bd-4283-96bf-fde2585a94ac admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:27:02.822760 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec6189e-06c2-44d7-8e1c-f295cf069ecb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f4bb9f989e4024a5da28dced646b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:27:02.824764 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec6189e-06c2-44d7-8e1c-f295cf069ecb service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:02.825875 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09c0372-e3bd-4283-96bf-fde2585a94ac admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:27:02.826149 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec6189e-06c2-44d7-8e1c-f295cf069ecb service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:02.827629 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec6189e-06c2-44d7-8e1c-f295cf069ecb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:02.832223 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f599b430-cad8-4084-80e6-29ed4225aaaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:27:02.833148 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f599b430-cad8-4084-80e6-29ed4225aaaf admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:02.833765 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f599b430-cad8-4084-80e6-29ed4225aaaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:02.833765 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f599b430-cad8-4084-80e6-29ed4225aaaf admin admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:02.834392 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f599b430-cad8-4084-80e6-29ed4225aaaf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:27:02.837336 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f599b430-cad8-4084-80e6-29ed4225aaaf admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:27:02.867717 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec6189e-06c2-44d7-8e1c-f295cf069ecb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:27:02.871414 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec6189e-06c2-44d7-8e1c-f295cf069ecb service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:27:02.930843 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1109/2212] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:27:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:27:02.967300 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1104/2213] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jun 29 11:27:02 2022] POST /identity/v3/projects => generated 385 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:27:02.974706 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec6189e-06c2-44d7-8e1c-f295cf069ecb service neutron] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:27:02.987803 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec6189e-06c2-44d7-8e1c-f295cf069ecb service neutron] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:27:03.118797 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3898c6e5-e66c-4a80-a521-440e1223b756 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:27:03.119665 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3898c6e5-e66c-4a80-a521-440e1223b756 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:03.121585 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3898c6e5-e66c-4a80-a521-440e1223b756 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:03.121951 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3898c6e5-e66c-4a80-a521-440e1223b756 admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:03.122723 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3898c6e5-e66c-4a80-a521-440e1223b756 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:27:03.125629 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3898c6e5-e66c-4a80-a521-440e1223b756 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:27:03.150325 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:27:03.236309 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1110/2214] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jun 29 11:27:02 2022] POST /identity/v3/users => generated 293 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:27:03.240788 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f599b430-cad8-4084-80e6-29ed4225aaaf admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:27:03.267078 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f599b430-cad8-4084-80e6-29ed4225aaaf admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:27:03.276449 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4abf32b-9df6-4303-aaf5-30f6c87d5bd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:27:03.277436 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4abf32b-9df6-4303-aaf5-30f6c87d5bd0 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:03.278141 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4abf32b-9df6-4303-aaf5-30f6c87d5bd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:03.279330 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4abf32b-9df6-4303-aaf5-30f6c87d5bd0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:03.280258 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4abf32b-9df6-4303-aaf5-30f6c87d5bd0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:27:03.282073 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4abf32b-9df6-4303-aaf5-30f6c87d5bd0 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:27:03.297876 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1105/2215] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:27:03 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 11:27:03.303792 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3898c6e5-e66c-4a80-a521-440e1223b756 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:27:03.318795 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3898c6e5-e66c-4a80-a521-440e1223b756 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:27:03.328724 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d310d71b-90c2-4f76-89d1-42101fda964a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:27:03.329465 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d310d71b-90c2-4f76-89d1-42101fda964a admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:03.329820 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d310d71b-90c2-4f76-89d1-42101fda964a admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:03.330172 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d310d71b-90c2-4f76-89d1-42101fda964a admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:03.331680 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d310d71b-90c2-4f76-89d1-42101fda964a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:27:03.333514 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d310d71b-90c2-4f76-89d1-42101fda964a admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:27:03.352991 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1111/2216] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:27:03 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 11:27:03.353997 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4abf32b-9df6-4303-aaf5-30f6c87d5bd0 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:27:03.365233 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4abf32b-9df6-4303-aaf5-30f6c87d5bd0 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:27:03.370430 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96ccbd9-0708-4adb-a045-c9f06473de4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:27:03.371462 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96ccbd9-0708-4adb-a045-c9f06473de4b admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:03.371818 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96ccbd9-0708-4adb-a045-c9f06473de4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:03.372237 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96ccbd9-0708-4adb-a045-c9f06473de4b admin admin] PATH_INFO: `/v3/projects/21b7d3b502b241648b92a54dec2f6bfa/users/6b1cef0872b24d3cbca64ea32568ffd0/roles/e1c88eebe911432cab6a07fbeb84653b` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:03.388979 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96ccbd9-0708-4adb-a045-c9f06473de4b admin admin] RBAC: Authorizing `identity:create_grant(project_id=21b7d3b502b241648b92a54dec2f6bfa, user_id=6b1cef0872b24d3cbca64ea32568ffd0, role_id=e1c88eebe911432cab6a07fbeb84653b)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:27:03.392594 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96ccbd9-0708-4adb-a045-c9f06473de4b admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:27:03.477156 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1106/2217] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jun 29 11:27:03 2022] PUT /identity/v3/projects/21b7d3b502b241648b92a54dec2f6bfa/users/6b1cef0872b24d3cbca64ea32568ffd0/roles/e1c88eebe911432cab6a07fbeb84653b => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:27:03.483341 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8cf52d-222f-49c6-9d4f-93221ff19ca2 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:03.483666 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8cf52d-222f-49c6-9d4f-93221ff19ca2 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:03.483952 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8cf52d-222f-49c6-9d4f-93221ff19ca2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:03.522117 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-2b8cf52d-222f-49c6-9d4f-93221ff19ca2 None None] MFA Rules not processed for user `6b1cef0872b24d3cbca64ea32568ffd0`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:27:03.525497 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-2b8cf52d-222f-49c6-9d4f-93221ff19ca2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:27:03.621764 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1112/2218] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:27:03 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:27:03.638782 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96ccbd9-0708-4adb-a045-c9f06473de4b admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:27:03.640954 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-a96ccbd9-0708-4adb-a045-c9f06473de4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:27:03.680104 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8cf52d-222f-49c6-9d4f-93221ff19ca2 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:27:03.681715 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-2b8cf52d-222f-49c6-9d4f-93221ff19ca2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:27:03.714599 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96ccbd9-0708-4adb-a045-c9f06473de4b admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:27:03.719468 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42362ad-e0fb-477d-91ea-4bf4b3a6976b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff7931b75a924394b1af733febce0834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:27:03.720380 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42362ad-e0fb-477d-91ea-4bf4b3a6976b service cinder] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:03.720755 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42362ad-e0fb-477d-91ea-4bf4b3a6976b service cinder] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:03.720932 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42362ad-e0fb-477d-91ea-4bf4b3a6976b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:03.737497 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42362ad-e0fb-477d-91ea-4bf4b3a6976b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:27:03.741117 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42362ad-e0fb-477d-91ea-4bf4b3a6976b service cinder] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:27:03.759648 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1107/2219] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:27:03 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:27:03.786550 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8cf52d-222f-49c6-9d4f-93221ff19ca2 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:27:03.793463 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9940f9b9-20cc-487a-a928-d4c19114e58f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:27:03.794476 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9940f9b9-20cc-487a-a928-d4c19114e58f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:03.794661 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9940f9b9-20cc-487a-a928-d4c19114e58f admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:03.794938 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9940f9b9-20cc-487a-a928-d4c19114e58f admin admin] PATH_INFO: `/v3/projects/ce8bbb1b564245e1926ebf367203f2d7` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:03.796271 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9940f9b9-20cc-487a-a928-d4c19114e58f admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce8bbb1b564245e1926ebf367203f2d7)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:27:03.798606 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9940f9b9-20cc-487a-a928-d4c19114e58f admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:27:03.963054 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-9940f9b9-20cc-487a-a928-d4c19114e58f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce8bbb1b564245e1926ebf367203f2d7', 'request_id': 'req-9940f9b9-20cc-487a-a928-d4c19114e58f'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:27:03.970804 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-9940f9b9-20cc-487a-a928-d4c19114e58f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce8bbb1b564245e1926ebf367203f2d7', 'request_id': 'req-9940f9b9-20cc-487a-a928-d4c19114e58f'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:27:03.977887 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-9940f9b9-20cc-487a-a928-d4c19114e58f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce8bbb1b564245e1926ebf367203f2d7', 'request_id': 'req-9940f9b9-20cc-487a-a928-d4c19114e58f'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:27:04.069307 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-9940f9b9-20cc-487a-a928-d4c19114e58f admin admin] The token cache is being invalidate because project ce8bbb1b564245e1926ebf367203f2d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 11:27:04.069652 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-9940f9b9-20cc-487a-a928-d4c19114e58f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:27:04.070314 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-9940f9b9-20cc-487a-a928-d4c19114e58f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:27:04.072272 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1113/2220] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jun 29 11:27:03 2022] DELETE /identity/v3/projects/ce8bbb1b564245e1926ebf367203f2d7 => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 11:27:04.076906 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42362ad-e0fb-477d-91ea-4bf4b3a6976b service cinder] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:27:04.089279 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42362ad-e0fb-477d-91ea-4bf4b3a6976b service cinder] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:27:04.154505 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ba139d-5a5e-49e5-bcc5-006235d4f0d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:27:04.155548 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ba139d-5a5e-49e5-bcc5-006235d4f0d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:04.155855 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ba139d-5a5e-49e5-bcc5-006235d4f0d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:04.156148 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ba139d-5a5e-49e5-bcc5-006235d4f0d6 admin admin] PATH_INFO: `/v3/users/95040e2e4cba4b5f958ffafdc29ec4ae` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:04.157432 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ba139d-5a5e-49e5-bcc5-006235d4f0d6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=95040e2e4cba4b5f958ffafdc29ec4ae)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:27:04.159393 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ba139d-5a5e-49e5-bcc5-006235d4f0d6 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:27:04.677934 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-27ba139d-5a5e-49e5-bcc5-006235d4f0d6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '95040e2e4cba4b5f958ffafdc29ec4ae'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:27:04.758424 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-27ba139d-5a5e-49e5-bcc5-006235d4f0d6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '95040e2e4cba4b5f958ffafdc29ec4ae'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:27:04.759091 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-27ba139d-5a5e-49e5-bcc5-006235d4f0d6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '95040e2e4cba4b5f958ffafdc29ec4ae'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:27:04.781105 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-27ba139d-5a5e-49e5-bcc5-006235d4f0d6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '95040e2e4cba4b5f958ffafdc29ec4ae'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:27:04.782751 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-27ba139d-5a5e-49e5-bcc5-006235d4f0d6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '95040e2e4cba4b5f958ffafdc29ec4ae'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:27:04.807253 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1108/2221] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jun 29 11:27:04 2022] DELETE /identity/v3/users/95040e2e4cba4b5f958ffafdc29ec4ae => generated 0 bytes in 732 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:27:04.809680 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9940f9b9-20cc-487a-a928-d4c19114e58f admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:27:04.811407 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-9940f9b9-20cc-487a-a928-d4c19114e58f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:27:04.828203 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:27:04.832716 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9940f9b9-20cc-487a-a928-d4c19114e58f admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:27:04.983049 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcc0024-8872-424a-adee-37c751aeef0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:27:04.990153 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcc0024-8872-424a-adee-37c751aeef0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:04.990153 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcc0024-8872-424a-adee-37c751aeef0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:04.990153 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcc0024-8872-424a-adee-37c751aeef0f admin admin] PATH_INFO: `/v3/users/c2d7bd6b628949b085be11d67154a02e` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:04.990153 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcc0024-8872-424a-adee-37c751aeef0f admin admin] RBAC: Authorizing `identity:delete_user(user_id=c2d7bd6b628949b085be11d67154a02e)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:27:04.990153 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcc0024-8872-424a-adee-37c751aeef0f admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:27:05.059108 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7d1ada-a73c-40e8-a847-31be030af3a4 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:05.059108 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7d1ada-a73c-40e8-a847-31be030af3a4 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:05.059108 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7d1ada-a73c-40e8-a847-31be030af3a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:05.092080 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-7e7d1ada-a73c-40e8-a847-31be030af3a4 None None] MFA Rules not processed for user `275df17696084dce9e7af487275bacab`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:27:05.094116 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-7e7d1ada-a73c-40e8-a847-31be030af3a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:27:05.195825 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-abcc0024-8872-424a-adee-37c751aeef0f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c2d7bd6b628949b085be11d67154a02e'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:27:05.198071 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1109/2222] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:27:05 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 11:27:05.208241 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-abcc0024-8872-424a-adee-37c751aeef0f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c2d7bd6b628949b085be11d67154a02e'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:27:05.216411 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7d1ada-a73c-40e8-a847-31be030af3a4 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:27:05.218918 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-7e7d1ada-a73c-40e8-a847-31be030af3a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:27:05.295518 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7d1ada-a73c-40e8-a847-31be030af3a4 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:27:05.301756 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6f40d2-7c7b-4d2f-9e98-3bee0c64ef5c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '796aeeff16df4739bddec1f6dee61bfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:27:05.302417 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6f40d2-7c7b-4d2f-9e98-3bee0c64ef5c service nova] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:05.302729 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6f40d2-7c7b-4d2f-9e98-3bee0c64ef5c service nova] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:05.303063 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6f40d2-7c7b-4d2f-9e98-3bee0c64ef5c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:05.304988 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-fa6f40d2-7c7b-4d2f-9e98-3bee0c64ef5c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:27:05.317892 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6f40d2-7c7b-4d2f-9e98-3bee0c64ef5c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:27:05.319494 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6f40d2-7c7b-4d2f-9e98-3bee0c64ef5c service nova] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:27:05.329428 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1110/2223] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:27:05 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 11:27:05.335488 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-abcc0024-8872-424a-adee-37c751aeef0f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c2d7bd6b628949b085be11d67154a02e'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:27:05.343052 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-abcc0024-8872-424a-adee-37c751aeef0f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c2d7bd6b628949b085be11d67154a02e'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:27:05.343714 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-abcc0024-8872-424a-adee-37c751aeef0f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c2d7bd6b628949b085be11d67154a02e'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:27:05.365464 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6f40d2-7c7b-4d2f-9e98-3bee0c64ef5c service nova] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:27:05.367234 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-fa6f40d2-7c7b-4d2f-9e98-3bee0c64ef5c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:27:05.383485 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1114/2224] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jun 29 11:27:04 2022] DELETE /identity/v3/users/c2d7bd6b628949b085be11d67154a02e => generated 0 bytes in 574 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 11:27:05.388991 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110cb92b-cc4d-47e4-9846-5e462441c9cc None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:05.389439 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110cb92b-cc4d-47e4-9846-5e462441c9cc None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:05.395720 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110cb92b-cc4d-47e4-9846-5e462441c9cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:05.396818 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6f40d2-7c7b-4d2f-9e98-3bee0c64ef5c service nova] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:27:05.434356 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-110cb92b-cc4d-47e4-9846-5e462441c9cc None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:27:05.436516 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-110cb92b-cc4d-47e4-9846-5e462441c9cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:27:05.492575 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d09f18-cf4b-4d80-8615-31d932545efe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:27:05.493464 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d09f18-cf4b-4d80-8615-31d932545efe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:05.493746 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d09f18-cf4b-4d80-8615-31d932545efe admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:05.494019 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d09f18-cf4b-4d80-8615-31d932545efe admin admin] PATH_INFO: `/v3/projects/57d1c6bf51b54b3eb1113a4b2d3e9850` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:05.495368 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d09f18-cf4b-4d80-8615-31d932545efe admin admin] RBAC: Authorizing `identity:delete_project(project_id=57d1c6bf51b54b3eb1113a4b2d3e9850)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:27:05.498203 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d09f18-cf4b-4d80-8615-31d932545efe admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:27:05.537770 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=85,keystone:DELETE=10,keystone:INSERT=5 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:27:05.547392 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1115/2225] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:27:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:27:05.559332 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110cb92b-cc4d-47e4-9846-5e462441c9cc None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:27:05.561398 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-110cb92b-cc4d-47e4-9846-5e462441c9cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:27:05.618544 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110cb92b-cc4d-47e4-9846-5e462441c9cc None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:27:05.622278 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ea027d-069b-47c4-8f3f-7321e2aa23b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f4bb9f989e4024a5da28dced646b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:27:05.623518 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ea027d-069b-47c4-8f3f-7321e2aa23b4 service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:05.623806 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ea027d-069b-47c4-8f3f-7321e2aa23b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:05.624110 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ea027d-069b-47c4-8f3f-7321e2aa23b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:05.634454 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ea027d-069b-47c4-8f3f-7321e2aa23b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:27:05.635990 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ea027d-069b-47c4-8f3f-7321e2aa23b4 service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:27:05.645968 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1116/2226] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:27:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 11:27:05.700933 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-55d09f18-cf4b-4d80-8615-31d932545efe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '57d1c6bf51b54b3eb1113a4b2d3e9850', 'request_id': 'req-55d09f18-cf4b-4d80-8615-31d932545efe'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:27:05.821728 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-55d09f18-cf4b-4d80-8615-31d932545efe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '57d1c6bf51b54b3eb1113a4b2d3e9850', 'request_id': 'req-55d09f18-cf4b-4d80-8615-31d932545efe'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:27:05.828781 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-55d09f18-cf4b-4d80-8615-31d932545efe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '57d1c6bf51b54b3eb1113a4b2d3e9850', 'request_id': 'req-55d09f18-cf4b-4d80-8615-31d932545efe'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:27:05.839595 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-55d09f18-cf4b-4d80-8615-31d932545efe admin admin] The token cache is being invalidate because project 57d1c6bf51b54b3eb1113a4b2d3e9850 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 11:27:05.839947 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-55d09f18-cf4b-4d80-8615-31d932545efe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:27:05.840583 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-55d09f18-cf4b-4d80-8615-31d932545efe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:27:05.842962 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1111/2227] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jun 29 11:27:05 2022] DELETE /identity/v3/projects/57d1c6bf51b54b3eb1113a4b2d3e9850 => generated 0 bytes in 478 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:27:05.847927 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a09b28-5e0f-4ad2-b29c-0fdce4469a63 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:05.848401 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a09b28-5e0f-4ad2-b29c-0fdce4469a63 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:05.848544 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a09b28-5e0f-4ad2-b29c-0fdce4469a63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:05.869151 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-53a09b28-5e0f-4ad2-b29c-0fdce4469a63 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:27:05.871071 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-53a09b28-5e0f-4ad2-b29c-0fdce4469a63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:27:05.981276 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1117/2228] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:27:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:27:05.991210 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfa50f9-41b9-41db-bda5-a2742fcc1d4a None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:05.991503 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfa50f9-41b9-41db-bda5-a2742fcc1d4a None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:05.991786 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfa50f9-41b9-41db-bda5-a2742fcc1d4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:06.013469 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-dcfa50f9-41b9-41db-bda5-a2742fcc1d4a None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:27:06.014867 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-dcfa50f9-41b9-41db-bda5-a2742fcc1d4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:27:06.041345 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1112/2229] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:27:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:27:06.045470 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a09b28-5e0f-4ad2-b29c-0fdce4469a63 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:27:06.064222 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a09b28-5e0f-4ad2-b29c-0fdce4469a63 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:27:06.079261 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c76882-73a1-4aea-bf8d-f8017a059074 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:27:06.080050 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c76882-73a1-4aea-bf8d-f8017a059074 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:06.080505 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c76882-73a1-4aea-bf8d-f8017a059074 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:06.080999 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c76882-73a1-4aea-bf8d-f8017a059074 admin admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:06.081939 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c76882-73a1-4aea-bf8d-f8017a059074 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:27:06.082410 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c76882-73a1-4aea-bf8d-f8017a059074 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:27:06.084162 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c76882-73a1-4aea-bf8d-f8017a059074 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:27:06.097684 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1118/2230] 10.222.0.44 () {58 vars in 1152 bytes} [Wed Jun 29 11:27:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 11:27:06.102170 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfa50f9-41b9-41db-bda5-a2742fcc1d4a None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:27:06.112015 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfa50f9-41b9-41db-bda5-a2742fcc1d4a None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:27:06.116862 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6601ea07-ac2f-4d0d-9741-df44a4454eb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:27:06.117517 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6601ea07-ac2f-4d0d-9741-df44a4454eb4 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:06.117802 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6601ea07-ac2f-4d0d-9741-df44a4454eb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:06.118083 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6601ea07-ac2f-4d0d-9741-df44a4454eb4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:06.118784 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6601ea07-ac2f-4d0d-9741-df44a4454eb4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:27:06.120745 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6601ea07-ac2f-4d0d-9741-df44a4454eb4 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:27:06.193198 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c76882-73a1-4aea-bf8d-f8017a059074 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:27:06.194923 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-39c76882-73a1-4aea-bf8d-f8017a059074 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:27:06.209759 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c76882-73a1-4aea-bf8d-f8017a059074 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:27:06.215697 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba33b11-4b19-45a7-9e7c-d9cf035284b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:27:06.217231 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba33b11-4b19-45a7-9e7c-d9cf035284b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:06.220113 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba33b11-4b19-45a7-9e7c-d9cf035284b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:06.220113 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba33b11-4b19-45a7-9e7c-d9cf035284b7 admin admin] PATH_INFO: `/v3/projects/c60c0b7eceb549df90814f699bd7f129` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:06.220113 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba33b11-4b19-45a7-9e7c-d9cf035284b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c60c0b7eceb549df90814f699bd7f129)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:27:06.221860 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba33b11-4b19-45a7-9e7c-d9cf035284b7 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:27:06.244828 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1113/2231] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jun 29 11:27:06 2022] POST /identity/v3/projects => generated 381 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 11:27:06.246256 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6601ea07-ac2f-4d0d-9741-df44a4454eb4 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:27:06.341647 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6601ea07-ac2f-4d0d-9741-df44a4454eb4 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:27:06.356645 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-9ba33b11-4b19-45a7-9e7c-d9cf035284b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c60c0b7eceb549df90814f699bd7f129', 'request_id': 'req-9ba33b11-4b19-45a7-9e7c-d9cf035284b7'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:27:06.365895 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-9ba33b11-4b19-45a7-9e7c-d9cf035284b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c60c0b7eceb549df90814f699bd7f129', 'request_id': 'req-9ba33b11-4b19-45a7-9e7c-d9cf035284b7'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:27:06.372838 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-9ba33b11-4b19-45a7-9e7c-d9cf035284b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c60c0b7eceb549df90814f699bd7f129', 'request_id': 'req-9ba33b11-4b19-45a7-9e7c-d9cf035284b7'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:27:06.416376 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f535e327-0518-44be-a32f-d6795c2c382f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:27:06.417830 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f535e327-0518-44be-a32f-d6795c2c382f admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:06.418552 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f535e327-0518-44be-a32f-d6795c2c382f admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:06.419236 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f535e327-0518-44be-a32f-d6795c2c382f admin admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:06.420289 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f535e327-0518-44be-a32f-d6795c2c382f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:27:06.422853 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f535e327-0518-44be-a32f-d6795c2c382f admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:27:06.450005 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-9ba33b11-4b19-45a7-9e7c-d9cf035284b7 admin admin] The token cache is being invalidate because project c60c0b7eceb549df90814f699bd7f129 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 11:27:06.450856 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-9ba33b11-4b19-45a7-9e7c-d9cf035284b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:27:06.451490 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-9ba33b11-4b19-45a7-9e7c-d9cf035284b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:27:06.453086 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1119/2232] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jun 29 11:27:06 2022] DELETE /identity/v3/projects/c60c0b7eceb549df90814f699bd7f129 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 11:27:06.456819 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba33b11-4b19-45a7-9e7c-d9cf035284b7 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:27:06.458181 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-9ba33b11-4b19-45a7-9e7c-d9cf035284b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:27:06.471492 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba33b11-4b19-45a7-9e7c-d9cf035284b7 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:27:06.476065 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc20e855-6781-49cc-9670-314722615e8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:27:06.476901 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc20e855-6781-49cc-9670-314722615e8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:06.477182 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc20e855-6781-49cc-9670-314722615e8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:06.477453 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc20e855-6781-49cc-9670-314722615e8a admin admin] PATH_INFO: `/v3/users/df59f75672264d23bb649ac312afba47` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:06.478628 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc20e855-6781-49cc-9670-314722615e8a admin admin] RBAC: Authorizing `identity:delete_user(user_id=df59f75672264d23bb649ac312afba47)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:27:06.480585 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc20e855-6781-49cc-9670-314722615e8a admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:27:06.538652 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1114/2233] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jun 29 11:27:06 2022] POST /identity/v3/users => generated 291 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:27:06.540606 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f535e327-0518-44be-a32f-d6795c2c382f admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:27:06.546382 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-f535e327-0518-44be-a32f-d6795c2c382f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:27:06.569103 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f535e327-0518-44be-a32f-d6795c2c382f admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:27:06.581096 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605a200c-6dcb-49c3-8c91-a3f4d565dc45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:27:06.581797 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605a200c-6dcb-49c3-8c91-a3f4d565dc45 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:06.582085 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605a200c-6dcb-49c3-8c91-a3f4d565dc45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:06.587047 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605a200c-6dcb-49c3-8c91-a3f4d565dc45 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:06.587930 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605a200c-6dcb-49c3-8c91-a3f4d565dc45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:27:06.589598 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605a200c-6dcb-49c3-8c91-a3f4d565dc45 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:27:06.608051 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1115/2234] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:27:06 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 11:27:06.608641 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605a200c-6dcb-49c3-8c91-a3f4d565dc45 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:27:06.628126 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605a200c-6dcb-49c3-8c91-a3f4d565dc45 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:27:06.641124 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e003ef-8456-4b14-b361-a1679921b4ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:27:06.641971 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e003ef-8456-4b14-b361-a1679921b4ac admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:06.642272 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e003ef-8456-4b14-b361-a1679921b4ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:06.642627 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e003ef-8456-4b14-b361-a1679921b4ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:06.643476 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e003ef-8456-4b14-b361-a1679921b4ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:27:06.645138 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e003ef-8456-4b14-b361-a1679921b4ac admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:27:06.663930 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1116/2235] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:27:06 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 11:27:06.667686 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e003ef-8456-4b14-b361-a1679921b4ac admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:27:06.693413 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e003ef-8456-4b14-b361-a1679921b4ac admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:27:06.693846 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-fc20e855-6781-49cc-9670-314722615e8a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df59f75672264d23bb649ac312afba47'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:27:06.696095 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-fc20e855-6781-49cc-9670-314722615e8a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df59f75672264d23bb649ac312afba47'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:27:06.699137 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ef7602-8f2f-4d73-864f-739154573844 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:27:06.701042 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ef7602-8f2f-4d73-864f-739154573844 admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:06.702432 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ef7602-8f2f-4d73-864f-739154573844 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:06.702903 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ef7602-8f2f-4d73-864f-739154573844 admin admin] PATH_INFO: `/v3/projects/148412b1cd08409498dd83c4d4849676/users/b486162ddf324cc19e88f41ed984b0d3/roles/e1c88eebe911432cab6a07fbeb84653b` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:06.729436 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ef7602-8f2f-4d73-864f-739154573844 admin admin] RBAC: Authorizing `identity:create_grant(project_id=148412b1cd08409498dd83c4d4849676, user_id=b486162ddf324cc19e88f41ed984b0d3, role_id=e1c88eebe911432cab6a07fbeb84653b)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:27:06.735858 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ef7602-8f2f-4d73-864f-739154573844 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:27:06.812574 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-fc20e855-6781-49cc-9670-314722615e8a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df59f75672264d23bb649ac312afba47'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:27:06.820574 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-fc20e855-6781-49cc-9670-314722615e8a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df59f75672264d23bb649ac312afba47'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:27:06.821314 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-fc20e855-6781-49cc-9670-314722615e8a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df59f75672264d23bb649ac312afba47'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:27:06.844741 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1117/2236] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jun 29 11:27:06 2022] PUT /identity/v3/projects/148412b1cd08409498dd83c4d4849676/users/b486162ddf324cc19e88f41ed984b0d3/roles/e1c88eebe911432cab6a07fbeb84653b => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:27:06.856267 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998a2349-cfb1-4948-ac6e-cd389f569046 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:06.856978 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998a2349-cfb1-4948-ac6e-cd389f569046 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:06.857489 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998a2349-cfb1-4948-ac6e-cd389f569046 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:06.858367 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1120/2237] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jun 29 11:27:06 2022] DELETE /identity/v3/users/df59f75672264d23bb649ac312afba47 => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 11:27:06.896922 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-998a2349-cfb1-4948-ac6e-cd389f569046 None None] MFA Rules not processed for user `b486162ddf324cc19e88f41ed984b0d3`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:27:06.898397 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-998a2349-cfb1-4948-ac6e-cd389f569046 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:27:07.040219 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1118/2238] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:27:06 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 11:27:07.041276 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc20e855-6781-49cc-9670-314722615e8a admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:27:07.042818 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-fc20e855-6781-49cc-9670-314722615e8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:27:07.134759 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc20e855-6781-49cc-9670-314722615e8a admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:27:07.134759 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b289c4-9b43-43a1-adc8-e5723f379ab4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:27:07.134759 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b289c4-9b43-43a1-adc8-e5723f379ab4 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:07.134759 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b289c4-9b43-43a1-adc8-e5723f379ab4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:07.135149 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b289c4-9b43-43a1-adc8-e5723f379ab4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:07.135149 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b289c4-9b43-43a1-adc8-e5723f379ab4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:27:07.135149 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b289c4-9b43-43a1-adc8-e5723f379ab4 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:27:07.234521 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1121/2239] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jun 29 11:27:07 2022] POST /identity/v3/projects => generated 381 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:27:07.239671 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998a2349-cfb1-4948-ac6e-cd389f569046 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:27:07.261807 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998a2349-cfb1-4948-ac6e-cd389f569046 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:27:07.390132 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ba8de8-84bc-4759-9784-9634561cf0fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:27:07.390830 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ba8de8-84bc-4759-9784-9634561cf0fb admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:07.391170 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ba8de8-84bc-4759-9784-9634561cf0fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:07.391453 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ba8de8-84bc-4759-9784-9634561cf0fb admin admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:07.392235 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ba8de8-84bc-4759-9784-9634561cf0fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:27:07.394332 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ba8de8-84bc-4759-9784-9634561cf0fb admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:27:07.435609 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:27:07.543274 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1119/2240] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jun 29 11:27:07 2022] POST /identity/v3/users => generated 297 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:27:07.547914 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b289c4-9b43-43a1-adc8-e5723f379ab4 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:27:07.559394 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b289c4-9b43-43a1-adc8-e5723f379ab4 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:27:07.569549 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a0b7b8-4a41-4ef6-83af-86e41593c1f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:27:07.570416 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a0b7b8-4a41-4ef6-83af-86e41593c1f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:07.570767 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a0b7b8-4a41-4ef6-83af-86e41593c1f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:07.571114 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a0b7b8-4a41-4ef6-83af-86e41593c1f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:07.571946 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a0b7b8-4a41-4ef6-83af-86e41593c1f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:27:07.573927 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a0b7b8-4a41-4ef6-83af-86e41593c1f8 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:27:07.597016 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1122/2241] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:27:07 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 11:27:07.601507 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ba8de8-84bc-4759-9784-9634561cf0fb admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:27:07.616988 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ba8de8-84bc-4759-9784-9634561cf0fb admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:27:07.617847 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a0b7b8-4a41-4ef6-83af-86e41593c1f8 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:27:07.619527 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-d6a0b7b8-4a41-4ef6-83af-86e41593c1f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:27:07.622557 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cd9272-e839-4ba1-ab2b-23375a6897a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:27:07.623353 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cd9272-e839-4ba1-ab2b-23375a6897a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:07.623946 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cd9272-e839-4ba1-ab2b-23375a6897a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:07.623946 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cd9272-e839-4ba1-ab2b-23375a6897a6 admin admin] PATH_INFO: `/v3/projects/4e11c45552ac4e8c89cbe0cad8a43b47/users/54627890d7d2454c82de6f1ddd7f4324/roles/e95c2ce63e114cc19a9dda6ad21e794f` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:07.638626 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a0b7b8-4a41-4ef6-83af-86e41593c1f8 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:27:07.644099 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec73a9e6-f985-4e75-a9e2-6420f36ba048 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:27:07.645224 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec73a9e6-f985-4e75-a9e2-6420f36ba048 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:07.645698 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec73a9e6-f985-4e75-a9e2-6420f36ba048 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:07.646117 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec73a9e6-f985-4e75-a9e2-6420f36ba048 admin admin] PATH_INFO: `/v3/projects/0ef7281e9e0e4133867869ed0c286bc1` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:07.647585 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec73a9e6-f985-4e75-a9e2-6420f36ba048 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ef7281e9e0e4133867869ed0c286bc1)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:27:07.649905 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec73a9e6-f985-4e75-a9e2-6420f36ba048 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:27:07.652765 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cd9272-e839-4ba1-ab2b-23375a6897a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e11c45552ac4e8c89cbe0cad8a43b47, user_id=54627890d7d2454c82de6f1ddd7f4324, role_id=e95c2ce63e114cc19a9dda6ad21e794f)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:27:07.656160 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cd9272-e839-4ba1-ab2b-23375a6897a6 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:27:07.811263 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1120/2242] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jun 29 11:27:07 2022] PUT /identity/v3/projects/4e11c45552ac4e8c89cbe0cad8a43b47/users/54627890d7d2454c82de6f1ddd7f4324/roles/e95c2ce63e114cc19a9dda6ad21e794f => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:27:07.814623 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-ec73a9e6-f985-4e75-a9e2-6420f36ba048 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ef7281e9e0e4133867869ed0c286bc1', 'request_id': 'req-ec73a9e6-f985-4e75-a9e2-6420f36ba048'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:27:07.815165 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cd9272-e839-4ba1-ab2b-23375a6897a6 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:27:07.826441 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-ec73a9e6-f985-4e75-a9e2-6420f36ba048 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ef7281e9e0e4133867869ed0c286bc1', 'request_id': 'req-ec73a9e6-f985-4e75-a9e2-6420f36ba048'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:27:07.833721 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-ec73a9e6-f985-4e75-a9e2-6420f36ba048 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ef7281e9e0e4133867869ed0c286bc1', 'request_id': 'req-ec73a9e6-f985-4e75-a9e2-6420f36ba048'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:27:07.904248 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cd9272-e839-4ba1-ab2b-23375a6897a6 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:27:07.946015 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-ec73a9e6-f985-4e75-a9e2-6420f36ba048 admin admin] The token cache is being invalidate because project 0ef7281e9e0e4133867869ed0c286bc1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 11:27:07.946928 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-ec73a9e6-f985-4e75-a9e2-6420f36ba048 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:27:07.948111 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-ec73a9e6-f985-4e75-a9e2-6420f36ba048 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:27:07.956431 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1123/2243] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jun 29 11:27:07 2022] DELETE /identity/v3/projects/0ef7281e9e0e4133867869ed0c286bc1 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 11:27:07.967668 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93efbcf1-7c22-4b42-9f14-47e6e1f3cbc1 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:07.969674 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93efbcf1-7c22-4b42-9f14-47e6e1f3cbc1 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:07.969674 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93efbcf1-7c22-4b42-9f14-47e6e1f3cbc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:08.004912 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ea4c57-2570-48b7-ae31-fff68c0be7db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:27:08.005749 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ea4c57-2570-48b7-ae31-fff68c0be7db admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:08.006009 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ea4c57-2570-48b7-ae31-fff68c0be7db admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:08.006294 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ea4c57-2570-48b7-ae31-fff68c0be7db admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:08.007038 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ea4c57-2570-48b7-ae31-fff68c0be7db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:27:08.009937 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ea4c57-2570-48b7-ae31-fff68c0be7db admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:27:08.010883 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-93efbcf1-7c22-4b42-9f14-47e6e1f3cbc1 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:27:08.012610 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-93efbcf1-7c22-4b42-9f14-47e6e1f3cbc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:27:08.022448 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1124/2244] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:27:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:27:08.026119 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1121/2245] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:27:07 2022] GET /identity/v3/roles => generated 1420 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 11:27:08.027275 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93efbcf1-7c22-4b42-9f14-47e6e1f3cbc1 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:27:08.029337 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-93efbcf1-7c22-4b42-9f14-47e6e1f3cbc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:27:08.032425 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565418ba-3b96-4d49-be21-6a89378de715 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:08.032425 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565418ba-3b96-4d49-be21-6a89378de715 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:08.032425 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565418ba-3b96-4d49-be21-6a89378de715 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:08.045884 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93efbcf1-7c22-4b42-9f14-47e6e1f3cbc1 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:27:08.052709 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4da3e08-032e-42ca-a8db-e94648b1752c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:27:08.053742 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4da3e08-032e-42ca-a8db-e94648b1752c admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:08.054128 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4da3e08-032e-42ca-a8db-e94648b1752c admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:08.054504 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4da3e08-032e-42ca-a8db-e94648b1752c admin admin] PATH_INFO: `/v3/projects/4e11c45552ac4e8c89cbe0cad8a43b47/users/54627890d7d2454c82de6f1ddd7f4324/roles/e95c2ce63e114cc19a9dda6ad21e794f` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:08.057882 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4da3e08-032e-42ca-a8db-e94648b1752c admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e11c45552ac4e8c89cbe0cad8a43b47, user_id=54627890d7d2454c82de6f1ddd7f4324, role_id=e95c2ce63e114cc19a9dda6ad21e794f)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:27:08.062192 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4da3e08-032e-42ca-a8db-e94648b1752c admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:27:08.063295 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-565418ba-3b96-4d49-be21-6a89378de715 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:27:08.064561 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-565418ba-3b96-4d49-be21-6a89378de715 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:27:08.073868 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1122/2246] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:27:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:27:08.079984 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565418ba-3b96-4d49-be21-6a89378de715 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:27:08.085481 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1125/2247] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jun 29 11:27:08 2022] PUT /identity/v3/projects/4e11c45552ac4e8c89cbe0cad8a43b47/users/54627890d7d2454c82de6f1ddd7f4324/roles/e95c2ce63e114cc19a9dda6ad21e794f => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:27:08.087074 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0c8b57-bc68-4d08-ac70-ec342a0b6610 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:08.087615 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0c8b57-bc68-4d08-ac70-ec342a0b6610 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:08.088084 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0c8b57-bc68-4d08-ac70-ec342a0b6610 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:08.094639 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565418ba-3b96-4d49-be21-6a89378de715 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:27:08.119876 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-0f0c8b57-bc68-4d08-ac70-ec342a0b6610 None None] MFA Rules not processed for user `54627890d7d2454c82de6f1ddd7f4324`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:27:08.124203 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-0f0c8b57-bc68-4d08-ac70-ec342a0b6610 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:27:08.164342 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c61ea7-7dfd-4924-9026-8233309f688e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:27:08.165703 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c61ea7-7dfd-4924-9026-8233309f688e admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:08.165703 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c61ea7-7dfd-4924-9026-8233309f688e admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:08.165703 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c61ea7-7dfd-4924-9026-8233309f688e admin admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:08.166353 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c61ea7-7dfd-4924-9026-8233309f688e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:27:08.166667 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c61ea7-7dfd-4924-9026-8233309f688e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:27:08.168315 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c61ea7-7dfd-4924-9026-8233309f688e admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:27:08.195782 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1123/2248] 10.222.0.44 () {58 vars in 1152 bytes} [Wed Jun 29 11:27:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:27:08.196771 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c61ea7-7dfd-4924-9026-8233309f688e admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:27:08.212650 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c61ea7-7dfd-4924-9026-8233309f688e admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:27:08.226501 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1126/2249] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:27:08 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:27:08.227021 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736b0809-3660-4dec-86f1-c20083a1ef1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:27:08.227021 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736b0809-3660-4dec-86f1-c20083a1ef1e admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:08.227021 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736b0809-3660-4dec-86f1-c20083a1ef1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:08.227021 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736b0809-3660-4dec-86f1-c20083a1ef1e admin admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:08.227314 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736b0809-3660-4dec-86f1-c20083a1ef1e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:27:08.227314 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736b0809-3660-4dec-86f1-c20083a1ef1e admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:27:08.237160 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0c8b57-bc68-4d08-ac70-ec342a0b6610 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:27:08.238751 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-0f0c8b57-bc68-4d08-ac70-ec342a0b6610 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:27:08.304806 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0c8b57-bc68-4d08-ac70-ec342a0b6610 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:27:08.308752 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfdb5df-0524-412d-9d0a-65e474ae4d8f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff7931b75a924394b1af733febce0834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:27:08.309450 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfdb5df-0524-412d-9d0a-65e474ae4d8f service cinder] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:08.310279 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfdb5df-0524-412d-9d0a-65e474ae4d8f service cinder] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:08.311021 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfdb5df-0524-412d-9d0a-65e474ae4d8f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:08.321209 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfdb5df-0524-412d-9d0a-65e474ae4d8f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:27:08.322752 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfdb5df-0524-412d-9d0a-65e474ae4d8f service cinder] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:27:08.337326 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1127/2250] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:27:08 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 11:27:08.363789 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1124/2251] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jun 29 11:27:08 2022] POST /identity/v3/projects => generated 375 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:27:08.368345 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfdb5df-0524-412d-9d0a-65e474ae4d8f service cinder] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:27:08.380681 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfdb5df-0524-412d-9d0a-65e474ae4d8f service cinder] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:27:08.449207 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3af051c-f26e-4283-8d61-71bddebfad23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:27:08.449923 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3af051c-f26e-4283-8d61-71bddebfad23 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:08.450238 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3af051c-f26e-4283-8d61-71bddebfad23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:08.450507 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3af051c-f26e-4283-8d61-71bddebfad23 admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:08.451201 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3af051c-f26e-4283-8d61-71bddebfad23 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:27:08.453175 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3af051c-f26e-4283-8d61-71bddebfad23 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:27:08.546232 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736b0809-3660-4dec-86f1-c20083a1ef1e admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:27:08.559436 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1128/2252] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jun 29 11:27:08 2022] POST /identity/v3/users => generated 288 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:27:08.561227 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3af051c-f26e-4283-8d61-71bddebfad23 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:27:08.571020 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3af051c-f26e-4283-8d61-71bddebfad23 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:27:08.576004 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf92e41-f5a6-4cad-a857-f95273a7453a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:27:08.576711 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf92e41-f5a6-4cad-a857-f95273a7453a admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:08.576942 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf92e41-f5a6-4cad-a857-f95273a7453a admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:08.577217 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf92e41-f5a6-4cad-a857-f95273a7453a admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:08.578602 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf92e41-f5a6-4cad-a857-f95273a7453a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:27:08.580439 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf92e41-f5a6-4cad-a857-f95273a7453a admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:27:08.592802 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1129/2253] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:27:08 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 11:27:08.602699 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf92e41-f5a6-4cad-a857-f95273a7453a admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:27:08.614189 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf92e41-f5a6-4cad-a857-f95273a7453a admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:27:08.620265 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8be44db-f6f2-417b-9187-41175c8bb289 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:27:08.620781 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736b0809-3660-4dec-86f1-c20083a1ef1e admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:27:08.621112 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8be44db-f6f2-417b-9187-41175c8bb289 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:08.621469 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8be44db-f6f2-417b-9187-41175c8bb289 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:08.621817 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8be44db-f6f2-417b-9187-41175c8bb289 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:08.623275 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8be44db-f6f2-417b-9187-41175c8bb289 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:27:08.629966 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8be44db-f6f2-417b-9187-41175c8bb289 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:27:08.630090 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d459d3-eb09-4e34-9451-8134f0e427ec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff7931b75a924394b1af733febce0834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:27:08.630090 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d459d3-eb09-4e34-9451-8134f0e427ec service cinder] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:08.630090 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d459d3-eb09-4e34-9451-8134f0e427ec service cinder] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:08.630090 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d459d3-eb09-4e34-9451-8134f0e427ec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:08.632697 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-a0d459d3-eb09-4e34-9451-8134f0e427ec service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:27:08.637953 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1130/2254] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:27:08 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 11:27:08.642012 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8be44db-f6f2-417b-9187-41175c8bb289 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:27:08.652531 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:27:08.658782 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8be44db-f6f2-417b-9187-41175c8bb289 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:27:08.664714 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a24740-dc37-45b5-8d1a-4af2ff84aaa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:27:08.665624 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a24740-dc37-45b5-8d1a-4af2ff84aaa1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:08.666034 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a24740-dc37-45b5-8d1a-4af2ff84aaa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:08.666324 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a24740-dc37-45b5-8d1a-4af2ff84aaa1 admin admin] PATH_INFO: `/v3/projects/8d61366af696465c918cf0bd854cf50a/users/c64751bfb4d4410d965532ed4bffc51d/roles/e1c88eebe911432cab6a07fbeb84653b` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:08.683328 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a24740-dc37-45b5-8d1a-4af2ff84aaa1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d61366af696465c918cf0bd854cf50a, user_id=c64751bfb4d4410d965532ed4bffc51d, role_id=e1c88eebe911432cab6a07fbeb84653b)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:27:08.686635 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a24740-dc37-45b5-8d1a-4af2ff84aaa1 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:27:08.718722 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d459d3-eb09-4e34-9451-8134f0e427ec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:27:08.720691 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d459d3-eb09-4e34-9451-8134f0e427ec service cinder] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:27:08.732486 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1125/2255] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:27:08 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 11:27:08.737936 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fa6064-f2ba-45de-8d28-2db1a9022567 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:08.738805 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fa6064-f2ba-45de-8d28-2db1a9022567 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:08.739172 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fa6064-f2ba-45de-8d28-2db1a9022567 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:08.755983 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1131/2256] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jun 29 11:27:08 2022] PUT /identity/v3/projects/8d61366af696465c918cf0bd854cf50a/users/c64751bfb4d4410d965532ed4bffc51d/roles/e1c88eebe911432cab6a07fbeb84653b => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:27:08.756629 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a24740-dc37-45b5-8d1a-4af2ff84aaa1 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:27:08.758546 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-74a24740-dc37-45b5-8d1a-4af2ff84aaa1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:27:08.776815 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-36fa6064-f2ba-45de-8d28-2db1a9022567 None None] MFA Rules not processed for user `275df17696084dce9e7af487275bacab`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:27:08.782802 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-36fa6064-f2ba-45de-8d28-2db1a9022567 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:27:08.838132 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a24740-dc37-45b5-8d1a-4af2ff84aaa1 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:27:08.843778 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5826399e-93c8-4f41-9e48-d7e73d6e3a96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:27:08.845748 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5826399e-93c8-4f41-9e48-d7e73d6e3a96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:08.846100 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5826399e-93c8-4f41-9e48-d7e73d6e3a96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:08.846578 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5826399e-93c8-4f41-9e48-d7e73d6e3a96 admin admin] PATH_INFO: `/v3/users/6b1cef0872b24d3cbca64ea32568ffd0` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:08.847964 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5826399e-93c8-4f41-9e48-d7e73d6e3a96 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b1cef0872b24d3cbca64ea32568ffd0)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:27:08.849900 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5826399e-93c8-4f41-9e48-d7e73d6e3a96 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:27:08.852903 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1126/2257] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:27:08 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 11:27:08.858836 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bad7a0c-06a5-4ff9-99dd-1ed6290aa8cd None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:08.858836 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bad7a0c-06a5-4ff9-99dd-1ed6290aa8cd None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:08.858836 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bad7a0c-06a5-4ff9-99dd-1ed6290aa8cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:08.898388 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-7bad7a0c-06a5-4ff9-99dd-1ed6290aa8cd None None] MFA Rules not processed for user `c64751bfb4d4410d965532ed4bffc51d`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:27:08.899898 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-7bad7a0c-06a5-4ff9-99dd-1ed6290aa8cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:27:08.992818 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1127/2258] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:27:08 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:27:08.997566 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bad7a0c-06a5-4ff9-99dd-1ed6290aa8cd None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:27:09.013008 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bad7a0c-06a5-4ff9-99dd-1ed6290aa8cd None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:27:09.018321 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45b661e-4ede-4d6f-878d-cb79010cf74f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:27:09.019042 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45b661e-4ede-4d6f-878d-cb79010cf74f admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:09.019323 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45b661e-4ede-4d6f-878d-cb79010cf74f admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:09.019601 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45b661e-4ede-4d6f-878d-cb79010cf74f admin admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:09.020334 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45b661e-4ede-4d6f-878d-cb79010cf74f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:27:09.022549 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45b661e-4ede-4d6f-878d-cb79010cf74f admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:27:09.055339 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-5826399e-93c8-4f41-9e48-d7e73d6e3a96 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b1cef0872b24d3cbca64ea32568ffd0'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:27:09.055937 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-5826399e-93c8-4f41-9e48-d7e73d6e3a96 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b1cef0872b24d3cbca64ea32568ffd0'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:27:09.138612 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-5826399e-93c8-4f41-9e48-d7e73d6e3a96 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b1cef0872b24d3cbca64ea32568ffd0'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:27:09.151757 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-5826399e-93c8-4f41-9e48-d7e73d6e3a96 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b1cef0872b24d3cbca64ea32568ffd0'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:27:09.151757 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-5826399e-93c8-4f41-9e48-d7e73d6e3a96 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b1cef0872b24d3cbca64ea32568ffd0'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:27:09.151981 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1128/2259] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jun 29 11:27:08 2022] POST /identity/v3/projects => generated 373 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 11:27:09.156408 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45b661e-4ede-4d6f-878d-cb79010cf74f admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:27:09.160307 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-d45b661e-4ede-4d6f-878d-cb79010cf74f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:27:09.204624 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1132/2260] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jun 29 11:27:08 2022] DELETE /identity/v3/users/6b1cef0872b24d3cbca64ea32568ffd0 => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:27:09.207506 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d348ef-f689-4789-be08-66ca99275bdd None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:09.207810 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d348ef-f689-4789-be08-66ca99275bdd None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:09.208122 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d348ef-f689-4789-be08-66ca99275bdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:09.241126 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45b661e-4ede-4d6f-878d-cb79010cf74f admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:27:09.241625 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-56d348ef-f689-4789-be08-66ca99275bdd None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:27:09.243207 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-56d348ef-f689-4789-be08-66ca99275bdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:27:09.334014 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1133/2261] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:27:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:27:09.335600 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe90846-880d-4a25-87bb-44b8d17bc670 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:09.336109 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe90846-880d-4a25-87bb-44b8d17bc670 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:09.336542 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe90846-880d-4a25-87bb-44b8d17bc670 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:09.338555 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1134/2262] 10.222.0.44 () {54 vars in 912 bytes} [Wed Jun 29 11:27:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:27:09.339335 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe90846-880d-4a25-87bb-44b8d17bc670 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:27:09.339907 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da7571f-7387-475d-aabf-e9368bf2e8ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:27:09.341374 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da7571f-7387-475d-aabf-e9368bf2e8ff admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:09.341609 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-bfe90846-880d-4a25-87bb-44b8d17bc670 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:27:09.341949 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da7571f-7387-475d-aabf-e9368bf2e8ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:09.342410 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da7571f-7387-475d-aabf-e9368bf2e8ff admin admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:09.343984 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da7571f-7387-475d-aabf-e9368bf2e8ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:27:09.346378 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da7571f-7387-475d-aabf-e9368bf2e8ff admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:27:09.363029 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe90846-880d-4a25-87bb-44b8d17bc670 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:27:09.367780 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af12c86b-8258-4c2b-9836-193ead738658 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:27:09.368685 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af12c86b-8258-4c2b-9836-193ead738658 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:09.368972 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af12c86b-8258-4c2b-9836-193ead738658 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:09.369247 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af12c86b-8258-4c2b-9836-193ead738658 admin admin] PATH_INFO: `/v3/users/b486162ddf324cc19e88f41ed984b0d3` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:09.370592 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af12c86b-8258-4c2b-9836-193ead738658 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b486162ddf324cc19e88f41ed984b0d3)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:27:09.372596 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af12c86b-8258-4c2b-9836-193ead738658 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:27:09.429467 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1129/2263] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jun 29 11:27:09 2022] POST /identity/v3/users => generated 293 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 11:27:09.434060 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6354bda6-da68-435e-83c8-6cf5994be249 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:09.434360 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6354bda6-da68-435e-83c8-6cf5994be249 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:09.434663 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6354bda6-da68-435e-83c8-6cf5994be249 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:09.453046 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-6354bda6-da68-435e-83c8-6cf5994be249 None None] MFA Rules not processed for user `b5f4bb9f989e4024a5da28dced646b92`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:27:09.454248 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-6354bda6-da68-435e-83c8-6cf5994be249 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:27:09.506064 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1130/2264] 10.222.0.44 () {58 vars in 1009 bytes} [Wed Jun 29 11:27:09 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:27:09.512264 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6354bda6-da68-435e-83c8-6cf5994be249 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:27:09.514252 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-6354bda6-da68-435e-83c8-6cf5994be249 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:27:09.524209 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6354bda6-da68-435e-83c8-6cf5994be249 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:27:09.527852 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b681b64-5621-4736-8992-0153e69f2837 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f4bb9f989e4024a5da28dced646b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:27:09.528501 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b681b64-5621-4736-8992-0153e69f2837 service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:09.528786 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b681b64-5621-4736-8992-0153e69f2837 service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:09.529064 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b681b64-5621-4736-8992-0153e69f2837 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:09.538189 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b681b64-5621-4736-8992-0153e69f2837 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:27:09.539622 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b681b64-5621-4736-8992-0153e69f2837 service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:27:09.549310 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1131/2265] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:27:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:27:09.550198 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b681b64-5621-4736-8992-0153e69f2837 service neutron] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:27:09.559362 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b681b64-5621-4736-8992-0153e69f2837 service neutron] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:27:09.565288 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2242d6-2522-4a48-92f4-5d51fa6522df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:27:09.565928 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2242d6-2522-4a48-92f4-5d51fa6522df admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:09.566210 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2242d6-2522-4a48-92f4-5d51fa6522df admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:09.566481 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2242d6-2522-4a48-92f4-5d51fa6522df admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:09.567248 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2242d6-2522-4a48-92f4-5d51fa6522df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:27:09.568789 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2242d6-2522-4a48-92f4-5d51fa6522df admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:27:09.583024 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1132/2266] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:27:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 11:27:09.583024 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2242d6-2522-4a48-92f4-5d51fa6522df admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:27:09.583360 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-ed2242d6-2522-4a48-92f4-5d51fa6522df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:27:09.590971 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-af12c86b-8258-4c2b-9836-193ead738658 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b486162ddf324cc19e88f41ed984b0d3'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:27:09.591660 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-af12c86b-8258-4c2b-9836-193ead738658 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b486162ddf324cc19e88f41ed984b0d3'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:27:09.597795 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2242d6-2522-4a48-92f4-5d51fa6522df admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:27:09.604890 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080cc470-0c3e-4f53-ba20-d2a8142f7a19 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f4bb9f989e4024a5da28dced646b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:27:09.604890 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080cc470-0c3e-4f53-ba20-d2a8142f7a19 service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:09.604890 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080cc470-0c3e-4f53-ba20-d2a8142f7a19 service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:09.604890 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080cc470-0c3e-4f53-ba20-d2a8142f7a19 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:09.608114 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-080cc470-0c3e-4f53-ba20-d2a8142f7a19 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:27:09.621085 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080cc470-0c3e-4f53-ba20-d2a8142f7a19 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:27:09.623206 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080cc470-0c3e-4f53-ba20-d2a8142f7a19 service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:27:09.642803 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1133/2267] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:27:09 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:27:09.645316 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080cc470-0c3e-4f53-ba20-d2a8142f7a19 service neutron] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:27:09.647444 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-080cc470-0c3e-4f53-ba20-d2a8142f7a19 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:27:09.662309 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080cc470-0c3e-4f53-ba20-d2a8142f7a19 service neutron] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:27:09.667026 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbf6ec9-4d37-4527-beda-329e40be8bca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:27:09.667943 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbf6ec9-4d37-4527-beda-329e40be8bca admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:09.668231 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbf6ec9-4d37-4527-beda-329e40be8bca admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:09.668513 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbf6ec9-4d37-4527-beda-329e40be8bca admin admin] PATH_INFO: `/v3/projects/af9fd3b990a549d6b4b1407d94c0ee7c/users/5da01b80e9f74de7bf58eb9ebdbc84db/roles/e95c2ce63e114cc19a9dda6ad21e794f` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:09.683250 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbf6ec9-4d37-4527-beda-329e40be8bca admin admin] RBAC: Authorizing `identity:create_grant(project_id=af9fd3b990a549d6b4b1407d94c0ee7c, user_id=5da01b80e9f74de7bf58eb9ebdbc84db, role_id=e95c2ce63e114cc19a9dda6ad21e794f)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:27:09.686503 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbf6ec9-4d37-4527-beda-329e40be8bca admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:27:09.764442 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-af12c86b-8258-4c2b-9836-193ead738658 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b486162ddf324cc19e88f41ed984b0d3'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:27:09.771802 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-af12c86b-8258-4c2b-9836-193ead738658 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b486162ddf324cc19e88f41ed984b0d3'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:27:09.772457 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-af12c86b-8258-4c2b-9836-193ead738658 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b486162ddf324cc19e88f41ed984b0d3'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:27:09.789585 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1135/2268] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jun 29 11:27:09 2022] DELETE /identity/v3/users/b486162ddf324cc19e88f41ed984b0d3 => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:27:09.794087 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37fbd0a-f9ee-4411-92d6-dc706e5f099b None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:09.794373 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37fbd0a-f9ee-4411-92d6-dc706e5f099b None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:09.794648 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37fbd0a-f9ee-4411-92d6-dc706e5f099b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:09.812484 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-a37fbd0a-f9ee-4411-92d6-dc706e5f099b None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:27:09.813702 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-a37fbd0a-f9ee-4411-92d6-dc706e5f099b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:27:09.840906 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1134/2269] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jun 29 11:27:09 2022] PUT /identity/v3/projects/af9fd3b990a549d6b4b1407d94c0ee7c/users/5da01b80e9f74de7bf58eb9ebdbc84db/roles/e95c2ce63e114cc19a9dda6ad21e794f => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:27:09.856165 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbf6ec9-4d37-4527-beda-329e40be8bca admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:27:09.870738 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:27:09.890106 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1136/2270] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:27:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:27:09.900941 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37fbd0a-f9ee-4411-92d6-dc706e5f099b None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:27:09.903532 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-a37fbd0a-f9ee-4411-92d6-dc706e5f099b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:27:09.960648 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbf6ec9-4d37-4527-beda-329e40be8bca admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:27:09.970491 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f095190-9775-4cad-85be-89830afa9c9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:27:09.971193 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f095190-9775-4cad-85be-89830afa9c9b admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:09.971568 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f095190-9775-4cad-85be-89830afa9c9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:09.971878 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f095190-9775-4cad-85be-89830afa9c9b admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:09.972891 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f095190-9775-4cad-85be-89830afa9c9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:27:09.978032 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f095190-9775-4cad-85be-89830afa9c9b admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:27:09.993298 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1135/2271] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:27:09 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 11:27:09.995178 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f095190-9775-4cad-85be-89830afa9c9b admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:27:10.005497 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37fbd0a-f9ee-4411-92d6-dc706e5f099b None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:27:10.010031 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f095190-9775-4cad-85be-89830afa9c9b admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:27:10.015394 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726eeff7-76f1-493e-9d01-6644bda963df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:27:10.016370 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726eeff7-76f1-493e-9d01-6644bda963df admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:10.017415 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726eeff7-76f1-493e-9d01-6644bda963df admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:10.017713 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726eeff7-76f1-493e-9d01-6644bda963df admin admin] PATH_INFO: `/v3/projects/af9fd3b990a549d6b4b1407d94c0ee7c/users/5da01b80e9f74de7bf58eb9ebdbc84db/roles/e95c2ce63e114cc19a9dda6ad21e794f` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:10.020388 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726eeff7-76f1-493e-9d01-6644bda963df admin admin] RBAC: Authorizing `identity:create_grant(project_id=af9fd3b990a549d6b4b1407d94c0ee7c, user_id=5da01b80e9f74de7bf58eb9ebdbc84db, role_id=e95c2ce63e114cc19a9dda6ad21e794f)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:27:10.024039 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726eeff7-76f1-493e-9d01-6644bda963df admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:27:10.070884 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1136/2272] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jun 29 11:27:09 2022] PUT /identity/v3/projects/af9fd3b990a549d6b4b1407d94c0ee7c/users/5da01b80e9f74de7bf58eb9ebdbc84db/roles/e95c2ce63e114cc19a9dda6ad21e794f => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:27:10.073186 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a45018d-16df-463c-97f1-f39ebe53004c None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:10.073415 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a45018d-16df-463c-97f1-f39ebe53004c None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:10.073783 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a45018d-16df-463c-97f1-f39ebe53004c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:10.084830 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea0630a-b387-4415-b53a-3c51218eb61d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f4bb9f989e4024a5da28dced646b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:27:10.085629 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea0630a-b387-4415-b53a-3c51218eb61d service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:10.091131 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea0630a-b387-4415-b53a-3c51218eb61d service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:10.091131 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea0630a-b387-4415-b53a-3c51218eb61d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:10.098983 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea0630a-b387-4415-b53a-3c51218eb61d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:27:10.100498 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea0630a-b387-4415-b53a-3c51218eb61d service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:27:10.105276 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-9a45018d-16df-463c-97f1-f39ebe53004c None None] MFA Rules not processed for user `5da01b80e9f74de7bf58eb9ebdbc84db`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:27:10.107590 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-9a45018d-16df-463c-97f1-f39ebe53004c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:27:10.115518 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1137/2273] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:27:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 11:27:10.181904 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1137/2274] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:27:10 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 11:27:10.194009 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea0630a-b387-4415-b53a-3c51218eb61d service neutron] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:27:10.195520 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-3ea0630a-b387-4415-b53a-3c51218eb61d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:27:10.206701 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a45018d-16df-463c-97f1-f39ebe53004c None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:27:10.209441 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-9a45018d-16df-463c-97f1-f39ebe53004c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:27:10.265285 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea0630a-b387-4415-b53a-3c51218eb61d service neutron] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:27:10.297971 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a45018d-16df-463c-97f1-f39ebe53004c None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:27:10.303630 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a42214-0059-4bd7-9c34-64c6509e4688 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:27:10.304523 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a42214-0059-4bd7-9c34-64c6509e4688 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:10.304812 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a42214-0059-4bd7-9c34-64c6509e4688 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:10.305097 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a42214-0059-4bd7-9c34-64c6509e4688 admin admin] PATH_INFO: `/v3/projects/21b7d3b502b241648b92a54dec2f6bfa` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:10.306382 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a42214-0059-4bd7-9c34-64c6509e4688 admin admin] RBAC: Authorizing `identity:delete_project(project_id=21b7d3b502b241648b92a54dec2f6bfa)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:27:10.308339 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a42214-0059-4bd7-9c34-64c6509e4688 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:27:10.334333 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28466e45-ec00-40f2-ac52-5f1005559c96 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff7931b75a924394b1af733febce0834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:27:10.335257 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28466e45-ec00-40f2-ac52-5f1005559c96 service cinder] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:10.336169 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28466e45-ec00-40f2-ac52-5f1005559c96 service cinder] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:10.336477 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28466e45-ec00-40f2-ac52-5f1005559c96 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:10.339368 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-28466e45-ec00-40f2-ac52-5f1005559c96 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:27:10.454121 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28466e45-ec00-40f2-ac52-5f1005559c96 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:27:10.457114 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28466e45-ec00-40f2-ac52-5f1005559c96 service cinder] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:27:10.462442 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-85a42214-0059-4bd7-9c34-64c6509e4688 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '21b7d3b502b241648b92a54dec2f6bfa', 'request_id': 'req-85a42214-0059-4bd7-9c34-64c6509e4688'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:27:10.476478 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1138/2275] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:27:10 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 280 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:27:10.496513 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:27:10.578007 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-85a42214-0059-4bd7-9c34-64c6509e4688 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '21b7d3b502b241648b92a54dec2f6bfa', 'request_id': 'req-85a42214-0059-4bd7-9c34-64c6509e4688'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:27:10.584846 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-85a42214-0059-4bd7-9c34-64c6509e4688 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '21b7d3b502b241648b92a54dec2f6bfa', 'request_id': 'req-85a42214-0059-4bd7-9c34-64c6509e4688'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:27:10.598106 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-85a42214-0059-4bd7-9c34-64c6509e4688 admin admin] The token cache is being invalidate because project 21b7d3b502b241648b92a54dec2f6bfa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 11:27:10.598398 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-85a42214-0059-4bd7-9c34-64c6509e4688 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:27:10.599068 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-85a42214-0059-4bd7-9c34-64c6509e4688 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:27:10.600821 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1138/2276] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jun 29 11:27:10 2022] DELETE /identity/v3/projects/21b7d3b502b241648b92a54dec2f6bfa => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:27:10.605688 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93acf8fe-02ad-45f9-bbeb-e5456dd6c0f0 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:10.610773 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93acf8fe-02ad-45f9-bbeb-e5456dd6c0f0 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:10.610773 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93acf8fe-02ad-45f9-bbeb-e5456dd6c0f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:10.634782 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-93acf8fe-02ad-45f9-bbeb-e5456dd6c0f0 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:27:10.636148 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-93acf8fe-02ad-45f9-bbeb-e5456dd6c0f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:27:10.697459 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a42214-0059-4bd7-9c34-64c6509e4688 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:27:10.699410 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-85a42214-0059-4bd7-9c34-64c6509e4688 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:27:10.715073 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1139/2277] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:27:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 11:27:10.727953 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f77371a-8a2a-4e8e-a03d-89b6f925348d None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:10.727953 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f77371a-8a2a-4e8e-a03d-89b6f925348d None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:10.727953 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f77371a-8a2a-4e8e-a03d-89b6f925348d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:10.746591 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-2f77371a-8a2a-4e8e-a03d-89b6f925348d None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:27:10.747916 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-2f77371a-8a2a-4e8e-a03d-89b6f925348d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:27:10.755778 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1140/2278] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:27:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:27:10.760005 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f77371a-8a2a-4e8e-a03d-89b6f925348d None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:27:10.769239 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f77371a-8a2a-4e8e-a03d-89b6f925348d None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:27:10.774014 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3974708c-6000-48f9-aa42-0b1f671f6b19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:27:10.774645 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3974708c-6000-48f9-aa42-0b1f671f6b19 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:10.774926 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3974708c-6000-48f9-aa42-0b1f671f6b19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:10.775200 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3974708c-6000-48f9-aa42-0b1f671f6b19 admin admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:10.775949 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3974708c-6000-48f9-aa42-0b1f671f6b19 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:27:10.776257 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3974708c-6000-48f9-aa42-0b1f671f6b19 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:27:10.777825 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3974708c-6000-48f9-aa42-0b1f671f6b19 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:27:10.783053 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a42214-0059-4bd7-9c34-64c6509e4688 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:27:10.787936 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dc8193-e8bb-4322-9885-1c891d78de03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:27:10.788814 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dc8193-e8bb-4322-9885-1c891d78de03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:10.789180 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dc8193-e8bb-4322-9885-1c891d78de03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:10.789372 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dc8193-e8bb-4322-9885-1c891d78de03 admin admin] PATH_INFO: `/v3/projects/148412b1cd08409498dd83c4d4849676` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:10.789932 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1141/2279] 10.222.0.44 () {58 vars in 1152 bytes} [Wed Jun 29 11:27:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 11:27:10.790563 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80dc8193-e8bb-4322-9885-1c891d78de03 admin admin] RBAC: Authorizing `identity:delete_project(project_id=148412b1cd08409498dd83c4d4849676)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:27:10.792514 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80dc8193-e8bb-4322-9885-1c891d78de03 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:27:10.794128 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3974708c-6000-48f9-aa42-0b1f671f6b19 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:27:10.803708 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3974708c-6000-48f9-aa42-0b1f671f6b19 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:27:10.812762 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9c1ec6-0a71-4190-829c-5cc1811dd248 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:27:10.813836 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9c1ec6-0a71-4190-829c-5cc1811dd248 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:10.814277 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9c1ec6-0a71-4190-829c-5cc1811dd248 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:10.814627 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9c1ec6-0a71-4190-829c-5cc1811dd248 admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:10.815491 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9c1ec6-0a71-4190-829c-5cc1811dd248 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:27:10.817561 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9c1ec6-0a71-4190-829c-5cc1811dd248 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:27:10.898405 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-80dc8193-e8bb-4322-9885-1c891d78de03 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '148412b1cd08409498dd83c4d4849676', 'request_id': 'req-80dc8193-e8bb-4322-9885-1c891d78de03'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:27:10.932330 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:27:10.939316 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1142/2280] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jun 29 11:27:10 2022] POST /identity/v3/projects => generated 375 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 11:27:10.943372 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9c1ec6-0a71-4190-829c-5cc1811dd248 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:27:11.005435 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9c1ec6-0a71-4190-829c-5cc1811dd248 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:27:11.010065 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc128b8-0c31-4b59-a4d7-bb70d324dcc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:27:11.010795 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc128b8-0c31-4b59-a4d7-bb70d324dcc6 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:11.011083 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc128b8-0c31-4b59-a4d7-bb70d324dcc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:11.011359 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc128b8-0c31-4b59-a4d7-bb70d324dcc6 admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:11.012069 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc128b8-0c31-4b59-a4d7-bb70d324dcc6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:27:11.014577 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc128b8-0c31-4b59-a4d7-bb70d324dcc6 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:27:11.034840 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-80dc8193-e8bb-4322-9885-1c891d78de03 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '148412b1cd08409498dd83c4d4849676', 'request_id': 'req-80dc8193-e8bb-4322-9885-1c891d78de03'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:27:11.049354 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-80dc8193-e8bb-4322-9885-1c891d78de03 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '148412b1cd08409498dd83c4d4849676', 'request_id': 'req-80dc8193-e8bb-4322-9885-1c891d78de03'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:27:11.060467 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-80dc8193-e8bb-4322-9885-1c891d78de03 admin admin] The token cache is being invalidate because project 148412b1cd08409498dd83c4d4849676 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 11:27:11.060756 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-80dc8193-e8bb-4322-9885-1c891d78de03 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:27:11.061462 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-80dc8193-e8bb-4322-9885-1c891d78de03 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:27:11.062998 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1139/2281] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jun 29 11:27:10 2022] DELETE /identity/v3/projects/148412b1cd08409498dd83c4d4849676 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:27:11.066927 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dc8193-e8bb-4322-9885-1c891d78de03 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:27:11.068598 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-80dc8193-e8bb-4322-9885-1c891d78de03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:27:11.120818 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1143/2282] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jun 29 11:27:10 2022] POST /identity/v3/users => generated 288 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 11:27:11.125389 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc128b8-0c31-4b59-a4d7-bb70d324dcc6 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:27:11.127042 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-ccc128b8-0c31-4b59-a4d7-bb70d324dcc6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:27:11.132379 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dc8193-e8bb-4322-9885-1c891d78de03 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:27:11.137105 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d457eda-e13e-4278-b284-b2d16392c546 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:27:11.138526 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d457eda-e13e-4278-b284-b2d16392c546 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:11.138809 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d457eda-e13e-4278-b284-b2d16392c546 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:11.139084 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d457eda-e13e-4278-b284-b2d16392c546 admin admin] PATH_INFO: `/v3/users/54627890d7d2454c82de6f1ddd7f4324` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:11.140332 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d457eda-e13e-4278-b284-b2d16392c546 admin admin] RBAC: Authorizing `identity:delete_user(user_id=54627890d7d2454c82de6f1ddd7f4324)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:27:11.140745 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc128b8-0c31-4b59-a4d7-bb70d324dcc6 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:27:11.142273 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d457eda-e13e-4278-b284-b2d16392c546 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:27:11.146334 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33908f3a-845d-4bbf-b036-5deb4d91ad68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:27:11.149456 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33908f3a-845d-4bbf-b036-5deb4d91ad68 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:11.149456 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33908f3a-845d-4bbf-b036-5deb4d91ad68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:11.149456 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33908f3a-845d-4bbf-b036-5deb4d91ad68 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:11.149456 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33908f3a-845d-4bbf-b036-5deb4d91ad68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:27:11.150727 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33908f3a-845d-4bbf-b036-5deb4d91ad68 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:27:11.162969 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1144/2283] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:27:11 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 11:27:11.167076 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33908f3a-845d-4bbf-b036-5deb4d91ad68 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:27:11.178423 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33908f3a-845d-4bbf-b036-5deb4d91ad68 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:27:11.183743 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ebdf38-b615-444f-a496-aa70e034f4e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:27:11.184358 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ebdf38-b615-444f-a496-aa70e034f4e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:11.185252 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ebdf38-b615-444f-a496-aa70e034f4e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:11.185534 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ebdf38-b615-444f-a496-aa70e034f4e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:11.186260 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ebdf38-b615-444f-a496-aa70e034f4e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:27:11.187932 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ebdf38-b615-444f-a496-aa70e034f4e4 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:27:11.198984 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1145/2284] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:27:11 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 11:27:11.203974 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ebdf38-b615-444f-a496-aa70e034f4e4 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:27:11.214005 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ebdf38-b615-444f-a496-aa70e034f4e4 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:27:11.221587 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1793fa58-e809-4a1c-8c86-b5fb7fec119b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:27:11.221587 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1793fa58-e809-4a1c-8c86-b5fb7fec119b admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:11.221587 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1793fa58-e809-4a1c-8c86-b5fb7fec119b admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:11.221587 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1793fa58-e809-4a1c-8c86-b5fb7fec119b admin admin] PATH_INFO: `/v3/projects/c7653e362c7d49a3ae386f137c41889c/users/907e9be3b1f54f17b8e3841130d41cc7/roles/e1c88eebe911432cab6a07fbeb84653b` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:11.235904 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1793fa58-e809-4a1c-8c86-b5fb7fec119b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7653e362c7d49a3ae386f137c41889c, user_id=907e9be3b1f54f17b8e3841130d41cc7, role_id=e1c88eebe911432cab6a07fbeb84653b)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:27:11.238530 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1793fa58-e809-4a1c-8c86-b5fb7fec119b admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:27:11.355381 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1146/2285] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jun 29 11:27:11 2022] PUT /identity/v3/projects/c7653e362c7d49a3ae386f137c41889c/users/907e9be3b1f54f17b8e3841130d41cc7/roles/e1c88eebe911432cab6a07fbeb84653b => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:27:11.365218 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69943d93-bbaf-42ec-bc35-1f631b9aa6a1 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:11.365507 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69943d93-bbaf-42ec-bc35-1f631b9aa6a1 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:11.365783 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69943d93-bbaf-42ec-bc35-1f631b9aa6a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:11.386896 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-2d457eda-e13e-4278-b284-b2d16392c546 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '54627890d7d2454c82de6f1ddd7f4324'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:27:11.393389 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-69943d93-bbaf-42ec-bc35-1f631b9aa6a1 None None] MFA Rules not processed for user `907e9be3b1f54f17b8e3841130d41cc7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:27:11.394701 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-69943d93-bbaf-42ec-bc35-1f631b9aa6a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:27:11.452273 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-2d457eda-e13e-4278-b284-b2d16392c546 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '54627890d7d2454c82de6f1ddd7f4324'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:27:11.452908 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-2d457eda-e13e-4278-b284-b2d16392c546 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '54627890d7d2454c82de6f1ddd7f4324'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:27:11.462976 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1147/2286] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:27:11 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 11:27:11.471904 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-2d457eda-e13e-4278-b284-b2d16392c546 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '54627890d7d2454c82de6f1ddd7f4324'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:27:11.472707 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-2d457eda-e13e-4278-b284-b2d16392c546 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '54627890d7d2454c82de6f1ddd7f4324'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:27:11.475271 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69943d93-bbaf-42ec-bc35-1f631b9aa6a1 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:27:11.476864 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-69943d93-bbaf-42ec-bc35-1f631b9aa6a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:27:11.487172 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1140/2287] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jun 29 11:27:11 2022] DELETE /identity/v3/users/54627890d7d2454c82de6f1ddd7f4324 => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 11:27:11.540624 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69943d93-bbaf-42ec-bc35-1f631b9aa6a1 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:27:11.605464 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb4804c-57d2-40f3-9e1d-427e51656960 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff7931b75a924394b1af733febce0834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:27:11.605464 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb4804c-57d2-40f3-9e1d-427e51656960 service cinder] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:11.605806 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb4804c-57d2-40f3-9e1d-427e51656960 service cinder] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:11.606239 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb4804c-57d2-40f3-9e1d-427e51656960 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:11.608770 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-8eb4804c-57d2-40f3-9e1d-427e51656960 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:27:11.667665 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb4804c-57d2-40f3-9e1d-427e51656960 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:27:11.669576 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb4804c-57d2-40f3-9e1d-427e51656960 service cinder] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:27:11.678239 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1148/2288] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:27:11 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:27:11.829435 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:27:11.979999 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d457eda-e13e-4278-b284-b2d16392c546 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:27:11.982405 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-2d457eda-e13e-4278-b284-b2d16392c546 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:27:12.051790 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d457eda-e13e-4278-b284-b2d16392c546 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:27:12.056502 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6002a996-f13d-449a-8380-55e8dec59157 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:27:12.057364 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6002a996-f13d-449a-8380-55e8dec59157 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:12.057644 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6002a996-f13d-449a-8380-55e8dec59157 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:12.057914 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6002a996-f13d-449a-8380-55e8dec59157 admin admin] PATH_INFO: `/v3/projects/4e11c45552ac4e8c89cbe0cad8a43b47` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:12.059240 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6002a996-f13d-449a-8380-55e8dec59157 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e11c45552ac4e8c89cbe0cad8a43b47)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:27:12.061244 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6002a996-f13d-449a-8380-55e8dec59157 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:27:12.183910 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-6002a996-f13d-449a-8380-55e8dec59157 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e11c45552ac4e8c89cbe0cad8a43b47', 'request_id': 'req-6002a996-f13d-449a-8380-55e8dec59157'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:27:12.285616 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-6002a996-f13d-449a-8380-55e8dec59157 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e11c45552ac4e8c89cbe0cad8a43b47', 'request_id': 'req-6002a996-f13d-449a-8380-55e8dec59157'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:27:12.298294 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-6002a996-f13d-449a-8380-55e8dec59157 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e11c45552ac4e8c89cbe0cad8a43b47', 'request_id': 'req-6002a996-f13d-449a-8380-55e8dec59157'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:27:12.320073 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-6002a996-f13d-449a-8380-55e8dec59157 admin admin] The token cache is being invalidate because project 4e11c45552ac4e8c89cbe0cad8a43b47 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 11:27:12.320248 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-6002a996-f13d-449a-8380-55e8dec59157 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:27:12.320949 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-6002a996-f13d-449a-8380-55e8dec59157 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:27:12.322743 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1141/2289] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jun 29 11:27:11 2022] DELETE /identity/v3/projects/4e11c45552ac4e8c89cbe0cad8a43b47 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:27:12.328186 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0dcfed-72ed-4bb7-aea7-ed0575a36362 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:12.328654 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0dcfed-72ed-4bb7-aea7-ed0575a36362 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:12.328775 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0dcfed-72ed-4bb7-aea7-ed0575a36362 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:12.352268 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-2d0dcfed-72ed-4bb7-aea7-ed0575a36362 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:27:12.353507 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-2d0dcfed-72ed-4bb7-aea7-ed0575a36362 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:27:12.420012 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1149/2290] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:27:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:27:12.430834 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602489d5-a6f9-4293-ad5d-a178f40b9f47 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:12.431349 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602489d5-a6f9-4293-ad5d-a178f40b9f47 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:12.432163 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602489d5-a6f9-4293-ad5d-a178f40b9f47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:12.459794 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-602489d5-a6f9-4293-ad5d-a178f40b9f47 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:27:12.463132 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-602489d5-a6f9-4293-ad5d-a178f40b9f47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:27:12.475327 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1142/2291] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:27:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:27:12.479672 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0dcfed-72ed-4bb7-aea7-ed0575a36362 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:27:12.490355 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0dcfed-72ed-4bb7-aea7-ed0575a36362 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:27:12.495207 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b32f8da-6ba0-4925-bc09-046767a93ae5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:27:12.499175 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b32f8da-6ba0-4925-bc09-046767a93ae5 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:12.499175 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b32f8da-6ba0-4925-bc09-046767a93ae5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:12.499175 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b32f8da-6ba0-4925-bc09-046767a93ae5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:12.499175 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b32f8da-6ba0-4925-bc09-046767a93ae5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:27:12.499175 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b32f8da-6ba0-4925-bc09-046767a93ae5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:27:12.499855 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b32f8da-6ba0-4925-bc09-046767a93ae5 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:27:12.511509 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1150/2292] 10.222.0.44 () {58 vars in 1152 bytes} [Wed Jun 29 11:27:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:27:12.517432 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602489d5-a6f9-4293-ad5d-a178f40b9f47 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:27:12.526962 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602489d5-a6f9-4293-ad5d-a178f40b9f47 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:27:12.532328 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174c8c09-4741-4d43-b076-7a2bb34d2445 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:27:12.532974 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174c8c09-4741-4d43-b076-7a2bb34d2445 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:12.533254 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174c8c09-4741-4d43-b076-7a2bb34d2445 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:12.533641 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174c8c09-4741-4d43-b076-7a2bb34d2445 admin admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:12.534328 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174c8c09-4741-4d43-b076-7a2bb34d2445 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:27:12.536271 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174c8c09-4741-4d43-b076-7a2bb34d2445 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:27:12.651285 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1143/2293] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jun 29 11:27:12 2022] POST /identity/v3/projects => generated 377 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:27:12.656244 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b32f8da-6ba0-4925-bc09-046767a93ae5 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:27:12.666870 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b32f8da-6ba0-4925-bc09-046767a93ae5 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:27:12.735952 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4966c236-70d1-4324-8bcb-70f69559f1b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:27:12.736625 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4966c236-70d1-4324-8bcb-70f69559f1b5 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:12.736906 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4966c236-70d1-4324-8bcb-70f69559f1b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:12.737180 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4966c236-70d1-4324-8bcb-70f69559f1b5 admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:12.737916 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4966c236-70d1-4324-8bcb-70f69559f1b5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:27:12.739808 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4966c236-70d1-4324-8bcb-70f69559f1b5 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:27:12.818827 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1151/2294] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jun 29 11:27:12 2022] POST /identity/v3/users => generated 289 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 11:27:12.822841 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174c8c09-4741-4d43-b076-7a2bb34d2445 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:27:12.834931 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174c8c09-4741-4d43-b076-7a2bb34d2445 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:27:12.840162 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8615786-625f-453b-b563-e8129322eea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:27:12.841328 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8615786-625f-453b-b563-e8129322eea7 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:12.845558 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8615786-625f-453b-b563-e8129322eea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:12.845558 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8615786-625f-453b-b563-e8129322eea7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:12.845558 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8615786-625f-453b-b563-e8129322eea7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:27:12.845558 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8615786-625f-453b-b563-e8129322eea7 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:27:12.855397 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1144/2295] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:27:12 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 11:27:12.859939 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4966c236-70d1-4324-8bcb-70f69559f1b5 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:27:12.871577 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4966c236-70d1-4324-8bcb-70f69559f1b5 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:27:12.883382 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b84e045-3c45-416f-802b-66957749c09e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:27:12.891556 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b84e045-3c45-416f-802b-66957749c09e admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:12.891556 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b84e045-3c45-416f-802b-66957749c09e admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:12.891556 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b84e045-3c45-416f-802b-66957749c09e admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:12.891556 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b84e045-3c45-416f-802b-66957749c09e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:27:12.891556 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b84e045-3c45-416f-802b-66957749c09e admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:27:12.898265 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1152/2296] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:27:12 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 11:27:12.902463 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8615786-625f-453b-b563-e8129322eea7 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:27:12.913433 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8615786-625f-453b-b563-e8129322eea7 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:27:12.919962 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc2baf0-16ac-416a-bad2-c10ca9adb0b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:27:12.921063 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc2baf0-16ac-416a-bad2-c10ca9adb0b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:12.921340 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc2baf0-16ac-416a-bad2-c10ca9adb0b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:12.921611 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc2baf0-16ac-416a-bad2-c10ca9adb0b9 admin admin] PATH_INFO: `/v3/projects/839ebdca87cf43139a1e6d9a983eeb08/users/56c3310d021643c8a031e7aa8cf57ebf/roles/e1c88eebe911432cab6a07fbeb84653b` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:12.940355 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:27:12.942922 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc2baf0-16ac-416a-bad2-c10ca9adb0b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=839ebdca87cf43139a1e6d9a983eeb08, user_id=56c3310d021643c8a031e7aa8cf57ebf, role_id=e1c88eebe911432cab6a07fbeb84653b)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:27:12.945771 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc2baf0-16ac-416a-bad2-c10ca9adb0b9 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:27:13.019085 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1145/2297] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jun 29 11:27:12 2022] PUT /identity/v3/projects/839ebdca87cf43139a1e6d9a983eeb08/users/56c3310d021643c8a031e7aa8cf57ebf/roles/e1c88eebe911432cab6a07fbeb84653b => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:27:13.029897 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38655371-a77f-4a40-a5e2-9018660330c3 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:13.030263 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38655371-a77f-4a40-a5e2-9018660330c3 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:13.030610 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38655371-a77f-4a40-a5e2-9018660330c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:13.058026 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-38655371-a77f-4a40-a5e2-9018660330c3 None None] MFA Rules not processed for user `56c3310d021643c8a031e7aa8cf57ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:27:13.059268 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-38655371-a77f-4a40-a5e2-9018660330c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:27:13.124728 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1153/2298] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:27:13 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 11:27:13.129152 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc2baf0-16ac-416a-bad2-c10ca9adb0b9 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:27:13.196001 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc2baf0-16ac-416a-bad2-c10ca9adb0b9 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:27:13.200944 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0b67e2-953a-4120-9514-f4fcf769b363 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:27:13.201610 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0b67e2-953a-4120-9514-f4fcf769b363 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:13.201901 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0b67e2-953a-4120-9514-f4fcf769b363 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:13.202185 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0b67e2-953a-4120-9514-f4fcf769b363 admin admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:13.202881 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0b67e2-953a-4120-9514-f4fcf769b363 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:27:13.204901 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0b67e2-953a-4120-9514-f4fcf769b363 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:27:13.313920 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1146/2299] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jun 29 11:27:13 2022] POST /identity/v3/projects => generated 375 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:27:13.315354 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38655371-a77f-4a40-a5e2-9018660330c3 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:27:13.326673 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38655371-a77f-4a40-a5e2-9018660330c3 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:27:13.423372 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1d77bf-069b-4757-9a2d-0f075103f3e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:27:13.424122 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1d77bf-069b-4757-9a2d-0f075103f3e3 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:13.424396 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1d77bf-069b-4757-9a2d-0f075103f3e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:13.424677 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1d77bf-069b-4757-9a2d-0f075103f3e3 admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:13.425389 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1d77bf-069b-4757-9a2d-0f075103f3e3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:27:13.429791 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1d77bf-069b-4757-9a2d-0f075103f3e3 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:27:13.463680 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:27:13.538066 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1154/2300] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jun 29 11:27:13 2022] POST /identity/v3/users => generated 288 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:27:13.542008 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0b67e2-953a-4120-9514-f4fcf769b363 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:27:13.552797 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0b67e2-953a-4120-9514-f4fcf769b363 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:27:13.557905 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5269d770-a68f-462b-ab34-af2c2e1e34d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:27:13.567926 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5269d770-a68f-462b-ab34-af2c2e1e34d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:13.567926 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5269d770-a68f-462b-ab34-af2c2e1e34d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:13.567926 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5269d770-a68f-462b-ab34-af2c2e1e34d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:13.567926 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5269d770-a68f-462b-ab34-af2c2e1e34d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:27:13.567926 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5269d770-a68f-462b-ab34-af2c2e1e34d9 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:27:13.572079 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1147/2301] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:27:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 11:27:13.576037 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1d77bf-069b-4757-9a2d-0f075103f3e3 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:27:13.586595 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1d77bf-069b-4757-9a2d-0f075103f3e3 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:27:13.590031 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:27:13.597913 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2872063b-a605-4a11-8ccb-0741c0d30c86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:27:13.598698 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2872063b-a605-4a11-8ccb-0741c0d30c86 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:13.599093 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2872063b-a605-4a11-8ccb-0741c0d30c86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:13.599441 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2872063b-a605-4a11-8ccb-0741c0d30c86 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:13.600881 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2872063b-a605-4a11-8ccb-0741c0d30c86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:27:13.602930 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2872063b-a605-4a11-8ccb-0741c0d30c86 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:27:13.615075 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1155/2302] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:27:13 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 11:27:13.618938 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5269d770-a68f-462b-ab34-af2c2e1e34d9 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:27:13.629404 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5269d770-a68f-462b-ab34-af2c2e1e34d9 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:27:13.634507 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96396b6c-d3e3-4cb6-81d5-6601b3394140 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:27:13.635395 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96396b6c-d3e3-4cb6-81d5-6601b3394140 admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:13.635680 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96396b6c-d3e3-4cb6-81d5-6601b3394140 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:13.635990 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96396b6c-d3e3-4cb6-81d5-6601b3394140 admin admin] PATH_INFO: `/v3/projects/f81258f452ef4bfc97450f8875db6a6c/users/81dfe07015bd4c58a64adf74fddf062e/roles/e1c88eebe911432cab6a07fbeb84653b` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:13.653017 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96396b6c-d3e3-4cb6-81d5-6601b3394140 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f81258f452ef4bfc97450f8875db6a6c, user_id=81dfe07015bd4c58a64adf74fddf062e, role_id=e1c88eebe911432cab6a07fbeb84653b)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:27:13.656549 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96396b6c-d3e3-4cb6-81d5-6601b3394140 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:27:13.736357 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1148/2303] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jun 29 11:27:13 2022] PUT /identity/v3/projects/f81258f452ef4bfc97450f8875db6a6c/users/81dfe07015bd4c58a64adf74fddf062e/roles/e1c88eebe911432cab6a07fbeb84653b => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:27:13.752425 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66420df1-fa44-4463-8d36-8206b17fd31e None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:13.752717 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66420df1-fa44-4463-8d36-8206b17fd31e None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:13.753027 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66420df1-fa44-4463-8d36-8206b17fd31e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:13.784653 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-66420df1-fa44-4463-8d36-8206b17fd31e None None] MFA Rules not processed for user `81dfe07015bd4c58a64adf74fddf062e`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:27:13.786042 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-66420df1-fa44-4463-8d36-8206b17fd31e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:27:13.864583 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1156/2304] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:27:13 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 11:27:13.869230 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96396b6c-d3e3-4cb6-81d5-6601b3394140 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:27:13.971453 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96396b6c-d3e3-4cb6-81d5-6601b3394140 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:27:13.988901 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28010d5e-d3df-45c6-b797-67bb261118dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:27:13.989583 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28010d5e-d3df-45c6-b797-67bb261118dd admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:13.989876 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28010d5e-d3df-45c6-b797-67bb261118dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:13.990150 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28010d5e-d3df-45c6-b797-67bb261118dd admin admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:13.990858 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28010d5e-d3df-45c6-b797-67bb261118dd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:27:13.993117 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28010d5e-d3df-45c6-b797-67bb261118dd admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:27:14.132965 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1149/2305] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jun 29 11:27:13 2022] POST /identity/v3/projects => generated 377 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:27:14.139159 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66420df1-fa44-4463-8d36-8206b17fd31e None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:27:14.155579 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66420df1-fa44-4463-8d36-8206b17fd31e None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:27:14.235133 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cb77a9-dacc-4baa-a5e1-e227856d237d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:27:14.235977 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cb77a9-dacc-4baa-a5e1-e227856d237d admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:14.236368 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cb77a9-dacc-4baa-a5e1-e227856d237d admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:14.236732 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cb77a9-dacc-4baa-a5e1-e227856d237d admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:14.237499 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cb77a9-dacc-4baa-a5e1-e227856d237d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:27:14.241514 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cb77a9-dacc-4baa-a5e1-e227856d237d admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:27:14.372720 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1157/2306] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jun 29 11:27:14 2022] POST /identity/v3/users => generated 295 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:27:14.376734 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28010d5e-d3df-45c6-b797-67bb261118dd admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:27:14.390303 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28010d5e-d3df-45c6-b797-67bb261118dd admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:27:14.401505 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda3ad4c-1fc9-4078-b620-fd11a30ca470 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:27:14.402312 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda3ad4c-1fc9-4078-b620-fd11a30ca470 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:14.402516 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda3ad4c-1fc9-4078-b620-fd11a30ca470 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:14.403136 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda3ad4c-1fc9-4078-b620-fd11a30ca470 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:14.404860 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda3ad4c-1fc9-4078-b620-fd11a30ca470 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:27:14.406496 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda3ad4c-1fc9-4078-b620-fd11a30ca470 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:27:14.426337 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cb77a9-dacc-4baa-a5e1-e227856d237d admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:27:14.436351 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1150/2307] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:27:14 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 11:27:14.442095 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cb77a9-dacc-4baa-a5e1-e227856d237d admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:27:14.460216 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce7ec3a-a589-43e2-b3fc-f8d9d00e6eba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:27:14.461133 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce7ec3a-a589-43e2-b3fc-f8d9d00e6eba admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:14.461405 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce7ec3a-a589-43e2-b3fc-f8d9d00e6eba admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:14.462430 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce7ec3a-a589-43e2-b3fc-f8d9d00e6eba admin admin] PATH_INFO: `/v3/projects/a70f42baac824e95be70eb103c352679/users/f82d37ea772945bfb53bc06d018a1d31/roles/e95c2ce63e114cc19a9dda6ad21e794f` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:14.483546 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce7ec3a-a589-43e2-b3fc-f8d9d00e6eba admin admin] RBAC: Authorizing `identity:create_grant(project_id=a70f42baac824e95be70eb103c352679, user_id=f82d37ea772945bfb53bc06d018a1d31, role_id=e95c2ce63e114cc19a9dda6ad21e794f)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:27:14.492003 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce7ec3a-a589-43e2-b3fc-f8d9d00e6eba admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:27:14.509126 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97c0e62-fac1-4147-9a1b-5b1b5fd62f68 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:14.509427 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97c0e62-fac1-4147-9a1b-5b1b5fd62f68 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:14.509712 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97c0e62-fac1-4147-9a1b-5b1b5fd62f68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:14.529781 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-a97c0e62-fac1-4147-9a1b-5b1b5fd62f68 None None] MFA Rules not processed for user `907e9be3b1f54f17b8e3841130d41cc7`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:27:14.531388 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-a97c0e62-fac1-4147-9a1b-5b1b5fd62f68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:27:14.589618 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1158/2308] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jun 29 11:27:14 2022] PUT /identity/v3/projects/a70f42baac824e95be70eb103c352679/users/f82d37ea772945bfb53bc06d018a1d31/roles/e95c2ce63e114cc19a9dda6ad21e794f => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:27:14.600970 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce7ec3a-a589-43e2-b3fc-f8d9d00e6eba admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:27:14.610281 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1151/2309] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:27:14 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 11:27:14.685298 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce7ec3a-a589-43e2-b3fc-f8d9d00e6eba admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:27:14.689074 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a03d81-07b9-4b0d-ba66-088d8284e524 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:27:14.689839 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a03d81-07b9-4b0d-ba66-088d8284e524 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:14.690130 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a03d81-07b9-4b0d-ba66-088d8284e524 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:14.690410 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a03d81-07b9-4b0d-ba66-088d8284e524 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:14.691200 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a03d81-07b9-4b0d-ba66-088d8284e524 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:27:14.695137 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a03d81-07b9-4b0d-ba66-088d8284e524 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:27:14.719284 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1159/2310] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:27:14 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 11:27:14.723233 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97c0e62-fac1-4147-9a1b-5b1b5fd62f68 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:27:14.736648 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97c0e62-fac1-4147-9a1b-5b1b5fd62f68 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:27:14.744114 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85a4162-f971-4147-b7f8-63fda8d92d2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:27:14.745022 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85a4162-f971-4147-b7f8-63fda8d92d2f admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:14.747535 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85a4162-f971-4147-b7f8-63fda8d92d2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:14.747535 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85a4162-f971-4147-b7f8-63fda8d92d2f admin admin] PATH_INFO: `/v3/projects/a70f42baac824e95be70eb103c352679/users/f82d37ea772945bfb53bc06d018a1d31/roles/e95c2ce63e114cc19a9dda6ad21e794f` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:14.749768 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85a4162-f971-4147-b7f8-63fda8d92d2f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a70f42baac824e95be70eb103c352679, user_id=f82d37ea772945bfb53bc06d018a1d31, role_id=e95c2ce63e114cc19a9dda6ad21e794f)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:27:14.753536 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85a4162-f971-4147-b7f8-63fda8d92d2f admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:27:14.765027 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a03d81-07b9-4b0d-ba66-088d8284e524 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:27:14.767328 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-53a03d81-07b9-4b0d-ba66-088d8284e524 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:27:14.769426 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1152/2311] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jun 29 11:27:14 2022] PUT /identity/v3/projects/a70f42baac824e95be70eb103c352679/users/f82d37ea772945bfb53bc06d018a1d31/roles/e95c2ce63e114cc19a9dda6ad21e794f => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 11:27:14.783214 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2e53a7-17f3-4f8c-b52e-92e775a75a8e None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:14.783585 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2e53a7-17f3-4f8c-b52e-92e775a75a8e None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:14.784130 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2e53a7-17f3-4f8c-b52e-92e775a75a8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:14.786866 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a03d81-07b9-4b0d-ba66-088d8284e524 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:27:14.812218 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:27:14.828485 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-ae2e53a7-17f3-4f8c-b52e-92e775a75a8e None None] MFA Rules not processed for user `f82d37ea772945bfb53bc06d018a1d31`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:27:14.830323 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-ae2e53a7-17f3-4f8c-b52e-92e775a75a8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:27:14.869555 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:27:14.884066 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f46785-c0ca-4fb1-8d86-55e65ce95117 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:27:14.885613 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f46785-c0ca-4fb1-8d86-55e65ce95117 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:14.886056 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f46785-c0ca-4fb1-8d86-55e65ce95117 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:14.886504 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f46785-c0ca-4fb1-8d86-55e65ce95117 admin admin] PATH_INFO: `/v3/users/c64751bfb4d4410d965532ed4bffc51d` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:14.890061 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f46785-c0ca-4fb1-8d86-55e65ce95117 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c64751bfb4d4410d965532ed4bffc51d)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:27:14.892482 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f46785-c0ca-4fb1-8d86-55e65ce95117 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:27:14.952561 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1153/2312] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:27:14 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 11:27:14.968306 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2e53a7-17f3-4f8c-b52e-92e775a75a8e None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:27:14.973318 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-ae2e53a7-17f3-4f8c-b52e-92e775a75a8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:27:15.057898 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2e53a7-17f3-4f8c-b52e-92e775a75a8e None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:27:15.062671 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5faa0ea-50a5-479d-a5f4-e0f52a15d53a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff7931b75a924394b1af733febce0834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:27:15.063257 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5faa0ea-50a5-479d-a5f4-e0f52a15d53a service cinder] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:15.063782 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5faa0ea-50a5-479d-a5f4-e0f52a15d53a service cinder] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:15.064188 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5faa0ea-50a5-479d-a5f4-e0f52a15d53a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:15.092271 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-82f46785-c0ca-4fb1-8d86-55e65ce95117 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c64751bfb4d4410d965532ed4bffc51d'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:27:15.093334 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-82f46785-c0ca-4fb1-8d86-55e65ce95117 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c64751bfb4d4410d965532ed4bffc51d'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:27:15.126448 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5faa0ea-50a5-479d-a5f4-e0f52a15d53a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:27:15.128062 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5faa0ea-50a5-479d-a5f4-e0f52a15d53a service cinder] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:27:15.138120 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1154/2313] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:27:14 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 11:27:15.189256 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-82f46785-c0ca-4fb1-8d86-55e65ce95117 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c64751bfb4d4410d965532ed4bffc51d'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:27:15.190742 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:27:15.196996 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-82f46785-c0ca-4fb1-8d86-55e65ce95117 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c64751bfb4d4410d965532ed4bffc51d'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:27:15.197884 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-82f46785-c0ca-4fb1-8d86-55e65ce95117 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c64751bfb4d4410d965532ed4bffc51d'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:27:15.231186 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1160/2314] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jun 29 11:27:14 2022] DELETE /identity/v3/users/c64751bfb4d4410d965532ed4bffc51d => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:27:15.233318 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cb8eaa-3f5e-45bd-b52e-a54ba055bc33 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:15.233612 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cb8eaa-3f5e-45bd-b52e-a54ba055bc33 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:15.233892 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cb8eaa-3f5e-45bd-b52e-a54ba055bc33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:15.259717 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-e0cb8eaa-3f5e-45bd-b52e-a54ba055bc33 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:27:15.270008 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-e0cb8eaa-3f5e-45bd-b52e-a54ba055bc33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:27:15.338336 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1155/2315] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:27:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 11:27:15.364254 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f46785-c0ca-4fb1-8d86-55e65ce95117 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:27:15.364254 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-82f46785-c0ca-4fb1-8d86-55e65ce95117 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:27:15.367592 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc404445-4984-4da2-bdc3-4ede0d4de604 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:15.367925 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc404445-4984-4da2-bdc3-4ede0d4de604 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:15.368211 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc404445-4984-4da2-bdc3-4ede0d4de604 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:15.392187 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-cc404445-4984-4da2-bdc3-4ede0d4de604 None None] MFA Rules not processed for user `8effcd6f6be64f14b3c8bc71f8866a99`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:27:15.393566 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-cc404445-4984-4da2-bdc3-4ede0d4de604 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:27:15.423436 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f46785-c0ca-4fb1-8d86-55e65ce95117 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:27:15.441759 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c12613-b8aa-4ccb-bbfc-29c020cb01d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f4bb9f989e4024a5da28dced646b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:27:15.442432 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c12613-b8aa-4ccb-bbfc-29c020cb01d6 service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:15.442954 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c12613-b8aa-4ccb-bbfc-29c020cb01d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:15.443257 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c12613-b8aa-4ccb-bbfc-29c020cb01d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:15.456725 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c12613-b8aa-4ccb-bbfc-29c020cb01d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:27:15.458547 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c12613-b8aa-4ccb-bbfc-29c020cb01d6 service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:27:15.466844 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1156/2316] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:27:15 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:27:15.469678 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1161/2317] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:27:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:27:15.865533 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:27:16.053509 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc404445-4984-4da2-bdc3-4ede0d4de604 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:27:16.055090 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-cc404445-4984-4da2-bdc3-4ede0d4de604 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:27:16.088462 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc404445-4984-4da2-bdc3-4ede0d4de604 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:27:16.095116 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c684c25-19ca-472f-b0d6-bbac0ad4942e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:27:16.096725 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c684c25-19ca-472f-b0d6-bbac0ad4942e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:16.096986 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c684c25-19ca-472f-b0d6-bbac0ad4942e admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:16.097307 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c684c25-19ca-472f-b0d6-bbac0ad4942e admin admin] PATH_INFO: `/v3/projects/8d61366af696465c918cf0bd854cf50a` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:16.098859 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c684c25-19ca-472f-b0d6-bbac0ad4942e admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d61366af696465c918cf0bd854cf50a)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:27:16.101343 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c684c25-19ca-472f-b0d6-bbac0ad4942e admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:27:16.280560 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-9c684c25-19ca-472f-b0d6-bbac0ad4942e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d61366af696465c918cf0bd854cf50a', 'request_id': 'req-9c684c25-19ca-472f-b0d6-bbac0ad4942e'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:27:16.347144 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-9c684c25-19ca-472f-b0d6-bbac0ad4942e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d61366af696465c918cf0bd854cf50a', 'request_id': 'req-9c684c25-19ca-472f-b0d6-bbac0ad4942e'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:27:16.356390 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-9c684c25-19ca-472f-b0d6-bbac0ad4942e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d61366af696465c918cf0bd854cf50a', 'request_id': 'req-9c684c25-19ca-472f-b0d6-bbac0ad4942e'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:27:16.367050 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-9c684c25-19ca-472f-b0d6-bbac0ad4942e admin admin] The token cache is being invalidate because project 8d61366af696465c918cf0bd854cf50a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 11:27:16.367341 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-9c684c25-19ca-472f-b0d6-bbac0ad4942e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:27:16.368099 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-9c684c25-19ca-472f-b0d6-bbac0ad4942e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:27:16.370874 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1157/2318] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jun 29 11:27:16 2022] DELETE /identity/v3/projects/8d61366af696465c918cf0bd854cf50a => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:27:16.376477 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c12613-b8aa-4ccb-bbfc-29c020cb01d6 service neutron] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:27:16.399532 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c12613-b8aa-4ccb-bbfc-29c020cb01d6 service neutron] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:27:16.518017 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5d6ff5-9f85-4d53-84d4-8c5203f4c865 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:27:16.519113 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5d6ff5-9f85-4d53-84d4-8c5203f4c865 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:16.519463 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5d6ff5-9f85-4d53-84d4-8c5203f4c865 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:16.519857 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5d6ff5-9f85-4d53-84d4-8c5203f4c865 admin admin] PATH_INFO: `/v3/users/5da01b80e9f74de7bf58eb9ebdbc84db` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:16.521135 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5d6ff5-9f85-4d53-84d4-8c5203f4c865 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5da01b80e9f74de7bf58eb9ebdbc84db)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:27:16.523542 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5d6ff5-9f85-4d53-84d4-8c5203f4c865 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:27:16.728331 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-1f5d6ff5-9f85-4d53-84d4-8c5203f4c865 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5da01b80e9f74de7bf58eb9ebdbc84db'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:27:16.728735 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-1f5d6ff5-9f85-4d53-84d4-8c5203f4c865 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5da01b80e9f74de7bf58eb9ebdbc84db'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:27:16.811965 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-1f5d6ff5-9f85-4d53-84d4-8c5203f4c865 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5da01b80e9f74de7bf58eb9ebdbc84db'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:27:16.812847 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:27:16.820489 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-1f5d6ff5-9f85-4d53-84d4-8c5203f4c865 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5da01b80e9f74de7bf58eb9ebdbc84db'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:27:16.821282 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-1f5d6ff5-9f85-4d53-84d4-8c5203f4c865 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5da01b80e9f74de7bf58eb9ebdbc84db'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:27:16.839686 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1162/2319] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jun 29 11:27:16 2022] DELETE /identity/v3/users/5da01b80e9f74de7bf58eb9ebdbc84db => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:27:17.333495 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c684c25-19ca-472f-b0d6-bbac0ad4942e admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:27:17.335017 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-9c684c25-19ca-472f-b0d6-bbac0ad4942e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:27:17.350640 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c684c25-19ca-472f-b0d6-bbac0ad4942e admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:27:17.427266 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88b4b5f-53ec-4cf8-a8cc-67766765f01b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:27:17.428286 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88b4b5f-53ec-4cf8-a8cc-67766765f01b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:17.428794 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88b4b5f-53ec-4cf8-a8cc-67766765f01b admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:17.429276 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88b4b5f-53ec-4cf8-a8cc-67766765f01b admin admin] PATH_INFO: `/v3/projects/af9fd3b990a549d6b4b1407d94c0ee7c` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:17.431062 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88b4b5f-53ec-4cf8-a8cc-67766765f01b admin admin] RBAC: Authorizing `identity:delete_project(project_id=af9fd3b990a549d6b4b1407d94c0ee7c)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:27:17.433790 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88b4b5f-53ec-4cf8-a8cc-67766765f01b admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:27:17.560559 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-a88b4b5f-53ec-4cf8-a8cc-67766765f01b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af9fd3b990a549d6b4b1407d94c0ee7c', 'request_id': 'req-a88b4b5f-53ec-4cf8-a8cc-67766765f01b'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:27:17.698523 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-a88b4b5f-53ec-4cf8-a8cc-67766765f01b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af9fd3b990a549d6b4b1407d94c0ee7c', 'request_id': 'req-a88b4b5f-53ec-4cf8-a8cc-67766765f01b'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:27:17.707541 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-a88b4b5f-53ec-4cf8-a8cc-67766765f01b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af9fd3b990a549d6b4b1407d94c0ee7c', 'request_id': 'req-a88b4b5f-53ec-4cf8-a8cc-67766765f01b'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:27:17.720495 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-a88b4b5f-53ec-4cf8-a8cc-67766765f01b admin admin] The token cache is being invalidate because project af9fd3b990a549d6b4b1407d94c0ee7c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 11:27:17.721019 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-a88b4b5f-53ec-4cf8-a8cc-67766765f01b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:27:17.721719 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-a88b4b5f-53ec-4cf8-a8cc-67766765f01b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:27:17.726517 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1158/2320] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jun 29 11:27:17 2022] DELETE /identity/v3/projects/af9fd3b990a549d6b4b1407d94c0ee7c => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:27:17.728279 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd31dcf-5705-4566-bf0e-558ea42003cb None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:17.728901 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd31dcf-5705-4566-bf0e-558ea42003cb None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:17.729197 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd31dcf-5705-4566-bf0e-558ea42003cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:17.754013 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-7bd31dcf-5705-4566-bf0e-558ea42003cb None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:27:17.755257 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-7bd31dcf-5705-4566-bf0e-558ea42003cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:27:17.814456 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1163/2321] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:27:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:27:17.822626 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dfc672-b818-4f3f-af51-af73123b0a71 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:17.822919 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dfc672-b818-4f3f-af51-af73123b0a71 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:17.823414 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dfc672-b818-4f3f-af51-af73123b0a71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:17.847620 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-e1dfc672-b818-4f3f-af51-af73123b0a71 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:27:17.848977 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-e1dfc672-b818-4f3f-af51-af73123b0a71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:27:17.858791 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:27:17.859913 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1159/2322] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:27:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:27:17.864677 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd31dcf-5705-4566-bf0e-558ea42003cb None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:27:17.874743 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd31dcf-5705-4566-bf0e-558ea42003cb None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:27:17.879417 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2a0b3c-bcfe-4d3d-922f-b722ce07097c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:27:17.880109 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2a0b3c-bcfe-4d3d-922f-b722ce07097c admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:17.880399 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2a0b3c-bcfe-4d3d-922f-b722ce07097c admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:17.880684 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2a0b3c-bcfe-4d3d-922f-b722ce07097c admin admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:17.881448 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2a0b3c-bcfe-4d3d-922f-b722ce07097c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:27:17.881762 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2a0b3c-bcfe-4d3d-922f-b722ce07097c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:27:17.883354 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2a0b3c-bcfe-4d3d-922f-b722ce07097c admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:27:17.895675 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1164/2323] 10.222.0.44 () {58 vars in 1152 bytes} [Wed Jun 29 11:27:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:27:17.900063 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dfc672-b818-4f3f-af51-af73123b0a71 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:27:17.909936 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dfc672-b818-4f3f-af51-af73123b0a71 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:27:17.915234 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03a126f-110b-487b-85a1-87cc67a22ef3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:27:17.916243 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03a126f-110b-487b-85a1-87cc67a22ef3 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:17.916699 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03a126f-110b-487b-85a1-87cc67a22ef3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:17.917246 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03a126f-110b-487b-85a1-87cc67a22ef3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:17.923389 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03a126f-110b-487b-85a1-87cc67a22ef3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:27:17.925473 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03a126f-110b-487b-85a1-87cc67a22ef3 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:27:18.027537 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1160/2324] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jun 29 11:27:17 2022] POST /identity/v3/projects => generated 377 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:27:18.032263 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2a0b3c-bcfe-4d3d-922f-b722ce07097c admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:27:18.042240 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2a0b3c-bcfe-4d3d-922f-b722ce07097c admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:27:18.097089 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f24580-b7ce-4c39-a568-8c07e1c862e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:27:18.097683 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f24580-b7ce-4c39-a568-8c07e1c862e2 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:18.097970 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f24580-b7ce-4c39-a568-8c07e1c862e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:18.098251 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f24580-b7ce-4c39-a568-8c07e1c862e2 admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:18.098936 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f24580-b7ce-4c39-a568-8c07e1c862e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:27:18.100961 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f24580-b7ce-4c39-a568-8c07e1c862e2 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:27:18.214302 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1165/2325] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jun 29 11:27:18 2022] POST /identity/v3/users => generated 289 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:27:18.219340 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03a126f-110b-487b-85a1-87cc67a22ef3 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:27:18.231093 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03a126f-110b-487b-85a1-87cc67a22ef3 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:27:18.236288 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47335a0c-24e3-4368-bef4-07750920dd79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:27:18.236938 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47335a0c-24e3-4368-bef4-07750920dd79 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:18.237220 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47335a0c-24e3-4368-bef4-07750920dd79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:18.237493 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47335a0c-24e3-4368-bef4-07750920dd79 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:18.238224 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47335a0c-24e3-4368-bef4-07750920dd79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:27:18.239809 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47335a0c-24e3-4368-bef4-07750920dd79 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:27:18.253500 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1161/2326] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:27:18 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 11:27:18.254878 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f24580-b7ce-4c39-a568-8c07e1c862e2 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:27:18.266393 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f24580-b7ce-4c39-a568-8c07e1c862e2 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:27:18.271030 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cd019d-59d9-493b-8152-8cfe814c445c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:27:18.271680 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cd019d-59d9-493b-8152-8cfe814c445c admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:18.278552 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cd019d-59d9-493b-8152-8cfe814c445c admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:18.278552 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cd019d-59d9-493b-8152-8cfe814c445c admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:18.278552 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cd019d-59d9-493b-8152-8cfe814c445c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:27:18.278552 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cd019d-59d9-493b-8152-8cfe814c445c admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:27:18.284826 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1166/2327] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:27:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 11:27:18.288971 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47335a0c-24e3-4368-bef4-07750920dd79 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:27:18.298852 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47335a0c-24e3-4368-bef4-07750920dd79 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:27:18.307134 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82e7cdd-7f1f-485b-93c5-4971e0796fee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:27:18.309236 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82e7cdd-7f1f-485b-93c5-4971e0796fee admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:18.309236 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82e7cdd-7f1f-485b-93c5-4971e0796fee admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:18.309236 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82e7cdd-7f1f-485b-93c5-4971e0796fee admin admin] PATH_INFO: `/v3/projects/917389e0ce224a0a8aaab3fc826a0214/users/d3f063fbca1f412fa65b888e09f0c59e/roles/e1c88eebe911432cab6a07fbeb84653b` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:18.323666 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82e7cdd-7f1f-485b-93c5-4971e0796fee admin admin] RBAC: Authorizing `identity:create_grant(project_id=917389e0ce224a0a8aaab3fc826a0214, user_id=d3f063fbca1f412fa65b888e09f0c59e, role_id=e1c88eebe911432cab6a07fbeb84653b)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:27:18.326854 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82e7cdd-7f1f-485b-93c5-4971e0796fee admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:27:18.404191 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cd019d-59d9-493b-8152-8cfe814c445c admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:27:18.406445 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-e3cd019d-59d9-493b-8152-8cfe814c445c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:27:18.415550 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1162/2328] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jun 29 11:27:18 2022] PUT /identity/v3/projects/917389e0ce224a0a8aaab3fc826a0214/users/d3f063fbca1f412fa65b888e09f0c59e/roles/e1c88eebe911432cab6a07fbeb84653b => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:27:18.428185 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba8e9ea-fecd-498d-8d76-405a01b708db None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:18.428473 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba8e9ea-fecd-498d-8d76-405a01b708db None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:18.428749 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba8e9ea-fecd-498d-8d76-405a01b708db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:18.455734 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-7ba8e9ea-fecd-498d-8d76-405a01b708db None None] MFA Rules not processed for user `d3f063fbca1f412fa65b888e09f0c59e`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:27:18.456999 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-7ba8e9ea-fecd-498d-8d76-405a01b708db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:27:18.465400 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cd019d-59d9-493b-8152-8cfe814c445c admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:27:18.489734 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:27:18.520697 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f2b4af-80ea-4e67-8a6e-3a914ae9d0ae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff7931b75a924394b1af733febce0834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:27:18.521343 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f2b4af-80ea-4e67-8a6e-3a914ae9d0ae service cinder] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:18.521625 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f2b4af-80ea-4e67-8a6e-3a914ae9d0ae service cinder] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:18.521898 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f2b4af-80ea-4e67-8a6e-3a914ae9d0ae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:18.523717 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-14f2b4af-80ea-4e67-8a6e-3a914ae9d0ae service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:27:18.527922 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1163/2329] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:27:18 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:27:18.539192 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba8e9ea-fecd-498d-8d76-405a01b708db None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:27:18.540625 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-7ba8e9ea-fecd-498d-8d76-405a01b708db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:27:18.552814 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba8e9ea-fecd-498d-8d76-405a01b708db None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:27:18.559899 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3ca839-df13-45d3-aa8a-c6ad02c96e0e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff7931b75a924394b1af733febce0834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:27:18.559899 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3ca839-df13-45d3-aa8a-c6ad02c96e0e service cinder] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:18.559899 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3ca839-df13-45d3-aa8a-c6ad02c96e0e service cinder] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:18.559899 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3ca839-df13-45d3-aa8a-c6ad02c96e0e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:18.568128 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3ca839-df13-45d3-aa8a-c6ad02c96e0e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:27:18.569521 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3ca839-df13-45d3-aa8a-c6ad02c96e0e service cinder] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:27:18.579925 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1164/2330] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:27:18 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 11:27:18.583177 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f2b4af-80ea-4e67-8a6e-3a914ae9d0ae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:27:18.588875 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f2b4af-80ea-4e67-8a6e-3a914ae9d0ae service cinder] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:27:18.593885 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1167/2331] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:27:18 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 11:27:18.641415 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:27:19.909048 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:27:19.960550 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:27:20.275255 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3ca839-df13-45d3-aa8a-c6ad02c96e0e service cinder] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:27:20.286884 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3ca839-df13-45d3-aa8a-c6ad02c96e0e service cinder] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:27:20.290942 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d965d692-3bd6-478c-81a3-3b5f6b117fe8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff7931b75a924394b1af733febce0834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:27:20.291573 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d965d692-3bd6-478c-81a3-3b5f6b117fe8 service cinder] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:20.291962 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d965d692-3bd6-478c-81a3-3b5f6b117fe8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:20.292324 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d965d692-3bd6-478c-81a3-3b5f6b117fe8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:20.294205 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-d965d692-3bd6-478c-81a3-3b5f6b117fe8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:27:20.328650 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:27:20.376812 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d965d692-3bd6-478c-81a3-3b5f6b117fe8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:27:20.378288 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d965d692-3bd6-478c-81a3-3b5f6b117fe8 service cinder] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:27:20.388123 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1165/2332] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:27:20 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:27:20.726444 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:27:20.812657 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:27:20.906977 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:27:20.929871 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:27:21.021265 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:27:21.044013 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:27:21.089785 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:27:21.112761 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:27:21.183669 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:27:21.257980 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:27:21.359595 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:27:21.514702 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:27:21.729508 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:27:21.797963 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:27:21.952005 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:27:21.980754 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:27:22.089076 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:27:22.123361 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:27:22.200141 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:27:22.314438 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:27:32.269541 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=77,keystone:INSERT=3 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:27:32.388576 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=54 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:27:38.079866 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f2b4af-80ea-4e67-8a6e-3a914ae9d0ae service cinder] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:27:38.081284 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-14f2b4af-80ea-4e67-8a6e-3a914ae9d0ae service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:27:38.157204 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f2b4af-80ea-4e67-8a6e-3a914ae9d0ae service cinder] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:27:38.162511 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ccc149-2c44-4606-9781-9d0fddf72b47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:27:38.163574 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ccc149-2c44-4606-9781-9d0fddf72b47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:38.164081 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ccc149-2c44-4606-9781-9d0fddf72b47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:38.164550 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ccc149-2c44-4606-9781-9d0fddf72b47 admin admin] PATH_INFO: `/v3/users/56c3310d021643c8a031e7aa8cf57ebf` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:38.166033 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ccc149-2c44-4606-9781-9d0fddf72b47 admin admin] RBAC: Authorizing `identity:delete_user(user_id=56c3310d021643c8a031e7aa8cf57ebf)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:27:38.168039 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ccc149-2c44-4606-9781-9d0fddf72b47 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:27:38.308488 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d965d692-3bd6-478c-81a3-3b5f6b117fe8 service cinder] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:27:38.309898 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-d965d692-3bd6-478c-81a3-3b5f6b117fe8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:27:38.324042 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d965d692-3bd6-478c-81a3-3b5f6b117fe8 service cinder] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:27:38.329337 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13319b13-4752-4cc0-96f5-5e0b443fb264 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:27:38.330149 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13319b13-4752-4cc0-96f5-5e0b443fb264 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:38.330429 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13319b13-4752-4cc0-96f5-5e0b443fb264 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:38.330701 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13319b13-4752-4cc0-96f5-5e0b443fb264 admin admin] PATH_INFO: `/v3/users/0303383b06be4fb0b976f194194919d0` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:38.332047 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13319b13-4752-4cc0-96f5-5e0b443fb264 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0303383b06be4fb0b976f194194919d0)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:27:38.333992 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13319b13-4752-4cc0-96f5-5e0b443fb264 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:27:38.368913 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-01ccc149-2c44-4606-9781-9d0fddf72b47 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '56c3310d021643c8a031e7aa8cf57ebf'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:27:38.369553 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-01ccc149-2c44-4606-9781-9d0fddf72b47 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '56c3310d021643c8a031e7aa8cf57ebf'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:27:38.440239 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-01ccc149-2c44-4606-9781-9d0fddf72b47 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '56c3310d021643c8a031e7aa8cf57ebf'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:27:38.453935 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-01ccc149-2c44-4606-9781-9d0fddf72b47 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '56c3310d021643c8a031e7aa8cf57ebf'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:27:38.454656 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-01ccc149-2c44-4606-9781-9d0fddf72b47 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '56c3310d021643c8a031e7aa8cf57ebf'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:27:38.472827 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1168/2333] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jun 29 11:27:38 2022] DELETE /identity/v3/users/56c3310d021643c8a031e7aa8cf57ebf => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:27:38.477120 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a387d08-b98e-43a2-b80d-b451cab77d19 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:38.477414 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a387d08-b98e-43a2-b80d-b451cab77d19 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:38.477694 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a387d08-b98e-43a2-b80d-b451cab77d19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:38.494755 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-3a387d08-b98e-43a2-b80d-b451cab77d19 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:27:38.512702 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-3a387d08-b98e-43a2-b80d-b451cab77d19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:27:38.545677 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-13319b13-4752-4cc0-96f5-5e0b443fb264 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0303383b06be4fb0b976f194194919d0'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:27:38.555870 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1169/2334] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:27:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 11:27:38.565413 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a387d08-b98e-43a2-b80d-b451cab77d19 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:27:38.566845 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-3a387d08-b98e-43a2-b80d-b451cab77d19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:27:38.601631 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:27:38.623177 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-13319b13-4752-4cc0-96f5-5e0b443fb264 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0303383b06be4fb0b976f194194919d0'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:27:38.624220 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-13319b13-4752-4cc0-96f5-5e0b443fb264 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0303383b06be4fb0b976f194194919d0'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:27:38.637627 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a387d08-b98e-43a2-b80d-b451cab77d19 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:27:38.642115 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c1a0df-fcc8-4e0d-baec-07e3843be1c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f4bb9f989e4024a5da28dced646b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:27:38.642304 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-13319b13-4752-4cc0-96f5-5e0b443fb264 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0303383b06be4fb0b976f194194919d0'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:27:38.642956 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c1a0df-fcc8-4e0d-baec-07e3843be1c9 service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:38.643076 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-13319b13-4752-4cc0-96f5-5e0b443fb264 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0303383b06be4fb0b976f194194919d0'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:27:38.643214 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c1a0df-fcc8-4e0d-baec-07e3843be1c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:38.643490 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c1a0df-fcc8-4e0d-baec-07e3843be1c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:38.645402 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-08c1a0df-fcc8-4e0d-baec-07e3843be1c9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:27:38.662853 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c1a0df-fcc8-4e0d-baec-07e3843be1c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:27:38.664627 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c1a0df-fcc8-4e0d-baec-07e3843be1c9 service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:27:38.664876 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1166/2335] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jun 29 11:27:38 2022] DELETE /identity/v3/users/0303383b06be4fb0b976f194194919d0 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 11:27:38.668831 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1053795a-a065-48c3-9bd4-4bbd8eb5c9a9 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:38.669229 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1053795a-a065-48c3-9bd4-4bbd8eb5c9a9 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:38.669621 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1053795a-a065-48c3-9bd4-4bbd8eb5c9a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:38.678782 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1170/2336] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:27:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 11:27:38.691307 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-1053795a-a065-48c3-9bd4-4bbd8eb5c9a9 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:27:38.692718 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-1053795a-a065-48c3-9bd4-4bbd8eb5c9a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:27:38.779025 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1167/2337] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:27:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 11:27:38.794397 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c1a0df-fcc8-4e0d-baec-07e3843be1c9 service neutron] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:27:38.796205 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-08c1a0df-fcc8-4e0d-baec-07e3843be1c9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:27:38.807952 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c1a0df-fcc8-4e0d-baec-07e3843be1c9 service neutron] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:27:38.883047 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cb4b5a-27de-4f7f-a80a-9700a5651c8d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f4bb9f989e4024a5da28dced646b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:27:38.883767 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cb4b5a-27de-4f7f-a80a-9700a5651c8d service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:38.884263 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cb4b5a-27de-4f7f-a80a-9700a5651c8d service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:38.884542 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cb4b5a-27de-4f7f-a80a-9700a5651c8d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:38.898560 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cb4b5a-27de-4f7f-a80a-9700a5651c8d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:27:38.900316 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cb4b5a-27de-4f7f-a80a-9700a5651c8d service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:27:38.912481 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1171/2338] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:27:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:27:39.285705 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1053795a-a065-48c3-9bd4-4bbd8eb5c9a9 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:27:39.287227 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-1053795a-a065-48c3-9bd4-4bbd8eb5c9a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:27:39.310534 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1053795a-a065-48c3-9bd4-4bbd8eb5c9a9 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:27:39.321493 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21df5576-39f4-4ca2-a170-3712ab5c8d6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:27:39.322362 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21df5576-39f4-4ca2-a170-3712ab5c8d6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:39.322652 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21df5576-39f4-4ca2-a170-3712ab5c8d6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:39.322930 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21df5576-39f4-4ca2-a170-3712ab5c8d6a admin admin] PATH_INFO: `/v3/projects/839ebdca87cf43139a1e6d9a983eeb08` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:39.324172 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21df5576-39f4-4ca2-a170-3712ab5c8d6a admin admin] RBAC: Authorizing `identity:delete_project(project_id=839ebdca87cf43139a1e6d9a983eeb08)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:27:39.326093 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21df5576-39f4-4ca2-a170-3712ab5c8d6a admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:27:39.425050 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cb4b5a-27de-4f7f-a80a-9700a5651c8d service neutron] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:27:39.426674 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-c2cb4b5a-27de-4f7f-a80a-9700a5651c8d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:27:39.443479 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-21df5576-39f4-4ca2-a170-3712ab5c8d6a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '839ebdca87cf43139a1e6d9a983eeb08', 'request_id': 'req-21df5576-39f4-4ca2-a170-3712ab5c8d6a'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:27:39.443668 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cb4b5a-27de-4f7f-a80a-9700a5651c8d service neutron] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:27:39.504369 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265c2784-a774-4ffb-beb2-8d2526ca04dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:27:39.505231 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265c2784-a774-4ffb-beb2-8d2526ca04dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:39.505516 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265c2784-a774-4ffb-beb2-8d2526ca04dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:39.505796 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265c2784-a774-4ffb-beb2-8d2526ca04dd admin admin] PATH_INFO: `/v3/projects/35f8b66a4a8f472ab1e8928cdd3fb6cd` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:39.506978 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265c2784-a774-4ffb-beb2-8d2526ca04dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=35f8b66a4a8f472ab1e8928cdd3fb6cd)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:27:39.508918 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265c2784-a774-4ffb-beb2-8d2526ca04dd admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:27:39.552701 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-21df5576-39f4-4ca2-a170-3712ab5c8d6a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '839ebdca87cf43139a1e6d9a983eeb08', 'request_id': 'req-21df5576-39f4-4ca2-a170-3712ab5c8d6a'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:27:39.560046 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-21df5576-39f4-4ca2-a170-3712ab5c8d6a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '839ebdca87cf43139a1e6d9a983eeb08', 'request_id': 'req-21df5576-39f4-4ca2-a170-3712ab5c8d6a'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:27:39.566192 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:27:39.572555 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-21df5576-39f4-4ca2-a170-3712ab5c8d6a admin admin] The token cache is being invalidate because project 839ebdca87cf43139a1e6d9a983eeb08 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 11:27:39.572926 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-21df5576-39f4-4ca2-a170-3712ab5c8d6a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:27:39.573992 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-21df5576-39f4-4ca2-a170-3712ab5c8d6a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:27:39.576646 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1168/2339] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jun 29 11:27:39 2022] DELETE /identity/v3/projects/839ebdca87cf43139a1e6d9a983eeb08 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:27:39.580541 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21df5576-39f4-4ca2-a170-3712ab5c8d6a admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:27:39.582540 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-21df5576-39f4-4ca2-a170-3712ab5c8d6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:27:39.601419 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21df5576-39f4-4ca2-a170-3712ab5c8d6a admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:27:39.615062 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7630b286-d8f1-4901-8f23-c7856911261e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:27:39.615062 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7630b286-d8f1-4901-8f23-c7856911261e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:39.615062 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7630b286-d8f1-4901-8f23-c7856911261e admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:39.615062 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7630b286-d8f1-4901-8f23-c7856911261e admin admin] PATH_INFO: `/v3/users/81dfe07015bd4c58a64adf74fddf062e` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:39.615713 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7630b286-d8f1-4901-8f23-c7856911261e admin admin] RBAC: Authorizing `identity:delete_user(user_id=81dfe07015bd4c58a64adf74fddf062e)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:27:39.618904 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7630b286-d8f1-4901-8f23-c7856911261e admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:27:39.646056 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-265c2784-a774-4ffb-beb2-8d2526ca04dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '35f8b66a4a8f472ab1e8928cdd3fb6cd', 'request_id': 'req-265c2784-a774-4ffb-beb2-8d2526ca04dd'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:27:39.653070 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-265c2784-a774-4ffb-beb2-8d2526ca04dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '35f8b66a4a8f472ab1e8928cdd3fb6cd', 'request_id': 'req-265c2784-a774-4ffb-beb2-8d2526ca04dd'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:27:39.660878 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-265c2784-a774-4ffb-beb2-8d2526ca04dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '35f8b66a4a8f472ab1e8928cdd3fb6cd', 'request_id': 'req-265c2784-a774-4ffb-beb2-8d2526ca04dd'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:27:39.666887 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:27:39.760207 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-265c2784-a774-4ffb-beb2-8d2526ca04dd admin admin] The token cache is being invalidate because project 35f8b66a4a8f472ab1e8928cdd3fb6cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 11:27:39.760501 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-265c2784-a774-4ffb-beb2-8d2526ca04dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:27:39.761120 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-265c2784-a774-4ffb-beb2-8d2526ca04dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:27:39.762694 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1172/2340] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jun 29 11:27:39 2022] DELETE /identity/v3/projects/35f8b66a4a8f472ab1e8928cdd3fb6cd => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 11:27:39.769266 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9835b3fa-bbc4-4591-89b0-9eff1ee1e566 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:39.769557 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9835b3fa-bbc4-4591-89b0-9eff1ee1e566 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:39.769832 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9835b3fa-bbc4-4591-89b0-9eff1ee1e566 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:39.788273 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-9835b3fa-bbc4-4591-89b0-9eff1ee1e566 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:27:39.789468 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-9835b3fa-bbc4-4591-89b0-9eff1ee1e566 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:27:39.865781 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1173/2341] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:27:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 11:27:39.867651 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28a3d9a-9d42-4822-8ca8-998dd2505c0b None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:39.867982 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28a3d9a-9d42-4822-8ca8-998dd2505c0b None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:39.868502 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28a3d9a-9d42-4822-8ca8-998dd2505c0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:39.875619 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-7630b286-d8f1-4901-8f23-c7856911261e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81dfe07015bd4c58a64adf74fddf062e'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:27:39.890781 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-b28a3d9a-9d42-4822-8ca8-998dd2505c0b None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:27:39.892049 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-b28a3d9a-9d42-4822-8ca8-998dd2505c0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:27:39.900202 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1174/2342] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:27:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:27:39.904381 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28a3d9a-9d42-4822-8ca8-998dd2505c0b None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:27:39.914495 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28a3d9a-9d42-4822-8ca8-998dd2505c0b None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:27:39.920003 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb51960-6dd0-4a00-9265-22305b4a71f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:27:39.920970 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb51960-6dd0-4a00-9265-22305b4a71f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:39.921258 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb51960-6dd0-4a00-9265-22305b4a71f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:39.921537 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb51960-6dd0-4a00-9265-22305b4a71f5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:39.922270 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb51960-6dd0-4a00-9265-22305b4a71f5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:27:39.922598 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb51960-6dd0-4a00-9265-22305b4a71f5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:27:39.924486 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb51960-6dd0-4a00-9265-22305b4a71f5 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:27:39.939806 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1175/2343] 10.222.0.44 () {58 vars in 1152 bytes} [Wed Jun 29 11:27:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 11:27:39.940888 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb51960-6dd0-4a00-9265-22305b4a71f5 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:27:39.950956 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb51960-6dd0-4a00-9265-22305b4a71f5 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:27:39.956127 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a124d7a0-4b05-4b3f-ba25-9922a2fc0f5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:27:39.956772 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a124d7a0-4b05-4b3f-ba25-9922a2fc0f5a admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:39.957054 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a124d7a0-4b05-4b3f-ba25-9922a2fc0f5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:39.957328 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a124d7a0-4b05-4b3f-ba25-9922a2fc0f5a admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:39.958019 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a124d7a0-4b05-4b3f-ba25-9922a2fc0f5a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:27:39.960317 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a124d7a0-4b05-4b3f-ba25-9922a2fc0f5a admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:27:39.960541 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-7630b286-d8f1-4901-8f23-c7856911261e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81dfe07015bd4c58a64adf74fddf062e'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:27:39.961275 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-7630b286-d8f1-4901-8f23-c7856911261e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81dfe07015bd4c58a64adf74fddf062e'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:27:39.986282 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-7630b286-d8f1-4901-8f23-c7856911261e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81dfe07015bd4c58a64adf74fddf062e'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:27:39.987153 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-7630b286-d8f1-4901-8f23-c7856911261e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81dfe07015bd4c58a64adf74fddf062e'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:27:40.000381 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1169/2344] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jun 29 11:27:39 2022] DELETE /identity/v3/users/81dfe07015bd4c58a64adf74fddf062e => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 11:27:40.045812 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1176/2345] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jun 29 11:27:39 2022] POST /identity/v3/projects => generated 379 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 11:27:40.049988 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7630b286-d8f1-4901-8f23-c7856911261e admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:27:40.051716 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-7630b286-d8f1-4901-8f23-c7856911261e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:27:40.150741 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7630b286-d8f1-4901-8f23-c7856911261e admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:27:40.241904 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5ca3ed-7548-4cea-9d18-fdb5253b9d1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:27:40.242723 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5ca3ed-7548-4cea-9d18-fdb5253b9d1d admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:40.243120 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5ca3ed-7548-4cea-9d18-fdb5253b9d1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:40.243576 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5ca3ed-7548-4cea-9d18-fdb5253b9d1d admin admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:40.244475 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5ca3ed-7548-4cea-9d18-fdb5253b9d1d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:27:40.246481 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5ca3ed-7548-4cea-9d18-fdb5253b9d1d admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:27:40.357128 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1170/2346] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jun 29 11:27:40 2022] POST /identity/v3/users => generated 290 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:27:40.358400 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a124d7a0-4b05-4b3f-ba25-9922a2fc0f5a admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:27:40.370835 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a124d7a0-4b05-4b3f-ba25-9922a2fc0f5a admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:27:40.380447 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3272d53-1fe9-4a56-beeb-bae72e4d5b90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:27:40.380447 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3272d53-1fe9-4a56-beeb-bae72e4d5b90 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:40.380447 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3272d53-1fe9-4a56-beeb-bae72e4d5b90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:40.380447 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3272d53-1fe9-4a56-beeb-bae72e4d5b90 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:40.380841 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3272d53-1fe9-4a56-beeb-bae72e4d5b90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:27:40.380841 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3272d53-1fe9-4a56-beeb-bae72e4d5b90 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:27:40.397896 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1177/2347] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:27:40 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 11:27:40.398710 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5ca3ed-7548-4cea-9d18-fdb5253b9d1d admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:27:40.407518 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:27:40.411926 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5ca3ed-7548-4cea-9d18-fdb5253b9d1d admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:27:40.423652 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ed1ac3-c679-471a-aecb-8fd895b871f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:27:40.424868 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ed1ac3-c679-471a-aecb-8fd895b871f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:40.425279 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ed1ac3-c679-471a-aecb-8fd895b871f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:40.425697 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ed1ac3-c679-471a-aecb-8fd895b871f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:40.426771 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ed1ac3-c679-471a-aecb-8fd895b871f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:27:40.429286 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ed1ac3-c679-471a-aecb-8fd895b871f5 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:27:40.446049 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1171/2348] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:27:40 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 11:27:40.450353 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3272d53-1fe9-4a56-beeb-bae72e4d5b90 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:27:40.466693 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3272d53-1fe9-4a56-beeb-bae72e4d5b90 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:27:40.473877 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad456ba-4af6-4b67-b78e-cb23d116948a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:27:40.478799 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad456ba-4af6-4b67-b78e-cb23d116948a admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:40.479453 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad456ba-4af6-4b67-b78e-cb23d116948a admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:40.480310 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad456ba-4af6-4b67-b78e-cb23d116948a admin admin] PATH_INFO: `/v3/projects/cdc0456bfa5a4619a96b41d3437fd82c/users/e91cc0b93f484f759233469ec5f37bb4/roles/e1c88eebe911432cab6a07fbeb84653b` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:40.505204 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad456ba-4af6-4b67-b78e-cb23d116948a admin admin] RBAC: Authorizing `identity:create_grant(project_id=cdc0456bfa5a4619a96b41d3437fd82c, user_id=e91cc0b93f484f759233469ec5f37bb4, role_id=e1c88eebe911432cab6a07fbeb84653b)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:27:40.508216 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad456ba-4af6-4b67-b78e-cb23d116948a admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:27:40.545015 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ed1ac3-c679-471a-aecb-8fd895b871f5 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:27:40.547002 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-73ed1ac3-c679-471a-aecb-8fd895b871f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:27:40.564749 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ed1ac3-c679-471a-aecb-8fd895b871f5 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:27:40.571605 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d397994d-ff61-4ef0-92e7-44ab36a72c7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:27:40.572306 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d397994d-ff61-4ef0-92e7-44ab36a72c7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:40.572646 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d397994d-ff61-4ef0-92e7-44ab36a72c7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:40.572866 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d397994d-ff61-4ef0-92e7-44ab36a72c7c admin admin] PATH_INFO: `/v3/projects/f81258f452ef4bfc97450f8875db6a6c` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:40.574962 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d397994d-ff61-4ef0-92e7-44ab36a72c7c admin admin] RBAC: Authorizing `identity:delete_project(project_id=f81258f452ef4bfc97450f8875db6a6c)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:27:40.577547 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d397994d-ff61-4ef0-92e7-44ab36a72c7c admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:27:40.631548 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1178/2349] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jun 29 11:27:40 2022] PUT /identity/v3/projects/cdc0456bfa5a4619a96b41d3437fd82c/users/e91cc0b93f484f759233469ec5f37bb4/roles/e1c88eebe911432cab6a07fbeb84653b => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:27:40.635328 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c709b971-c904-4177-9ef9-1f31f31ea4d9 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:40.635807 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c709b971-c904-4177-9ef9-1f31f31ea4d9 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:40.636317 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c709b971-c904-4177-9ef9-1f31f31ea4d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:40.654369 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-c709b971-c904-4177-9ef9-1f31f31ea4d9 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:27:40.658145 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-c709b971-c904-4177-9ef9-1f31f31ea4d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:27:40.723332 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1179/2350] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:27:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:27:40.733651 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c709b971-c904-4177-9ef9-1f31f31ea4d9 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:27:40.735265 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-c709b971-c904-4177-9ef9-1f31f31ea4d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:27:40.742306 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-d397994d-ff61-4ef0-92e7-44ab36a72c7c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f81258f452ef4bfc97450f8875db6a6c', 'request_id': 'req-d397994d-ff61-4ef0-92e7-44ab36a72c7c'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:27:40.759454 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:27:40.798310 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c709b971-c904-4177-9ef9-1f31f31ea4d9 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:27:40.826690 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-d397994d-ff61-4ef0-92e7-44ab36a72c7c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f81258f452ef4bfc97450f8875db6a6c', 'request_id': 'req-d397994d-ff61-4ef0-92e7-44ab36a72c7c'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:27:40.834481 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-d397994d-ff61-4ef0-92e7-44ab36a72c7c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f81258f452ef4bfc97450f8875db6a6c', 'request_id': 'req-d397994d-ff61-4ef0-92e7-44ab36a72c7c'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:27:40.853418 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-d397994d-ff61-4ef0-92e7-44ab36a72c7c admin admin] The token cache is being invalidate because project f81258f452ef4bfc97450f8875db6a6c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 11:27:40.853740 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-d397994d-ff61-4ef0-92e7-44ab36a72c7c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:27:40.854527 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-d397994d-ff61-4ef0-92e7-44ab36a72c7c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:27:40.856480 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1172/2351] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jun 29 11:27:40 2022] DELETE /identity/v3/projects/f81258f452ef4bfc97450f8875db6a6c => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:27:40.860770 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d397994d-ff61-4ef0-92e7-44ab36a72c7c admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:27:40.860925 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cde36c-1fa5-4332-aa37-fdf9f236b2d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f4bb9f989e4024a5da28dced646b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:27:40.861590 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cde36c-1fa5-4332-aa37-fdf9f236b2d2 service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:40.861883 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cde36c-1fa5-4332-aa37-fdf9f236b2d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:40.862026 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-d397994d-ff61-4ef0-92e7-44ab36a72c7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:27:40.862163 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cde36c-1fa5-4332-aa37-fdf9f236b2d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:40.864034 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-e8cde36c-1fa5-4332-aa37-fdf9f236b2d2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:27:40.942329 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d397994d-ff61-4ef0-92e7-44ab36a72c7c admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:27:40.951139 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cde36c-1fa5-4332-aa37-fdf9f236b2d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:27:40.953185 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cde36c-1fa5-4332-aa37-fdf9f236b2d2 service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:27:40.953665 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172ed855-769d-46d4-b93d-4bc497c73817 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:27:40.954820 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172ed855-769d-46d4-b93d-4bc497c73817 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:40.955295 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172ed855-769d-46d4-b93d-4bc497c73817 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:40.955886 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172ed855-769d-46d4-b93d-4bc497c73817 admin admin] PATH_INFO: `/v3/users/f82d37ea772945bfb53bc06d018a1d31` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:40.957546 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172ed855-769d-46d4-b93d-4bc497c73817 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f82d37ea772945bfb53bc06d018a1d31)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:27:40.959810 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172ed855-769d-46d4-b93d-4bc497c73817 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:27:40.980050 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1180/2352] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:27:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 11:27:41.239286 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-172ed855-769d-46d4-b93d-4bc497c73817 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f82d37ea772945bfb53bc06d018a1d31'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:27:41.307075 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-172ed855-769d-46d4-b93d-4bc497c73817 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f82d37ea772945bfb53bc06d018a1d31'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:27:41.307713 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-172ed855-769d-46d4-b93d-4bc497c73817 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f82d37ea772945bfb53bc06d018a1d31'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:27:41.324652 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-172ed855-769d-46d4-b93d-4bc497c73817 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f82d37ea772945bfb53bc06d018a1d31'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:27:41.325749 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-172ed855-769d-46d4-b93d-4bc497c73817 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f82d37ea772945bfb53bc06d018a1d31'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:27:41.340477 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1173/2353] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jun 29 11:27:40 2022] DELETE /identity/v3/users/f82d37ea772945bfb53bc06d018a1d31 => generated 0 bytes in 480 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 11:27:41.426604 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:27:41.947788 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cde36c-1fa5-4332-aa37-fdf9f236b2d2 service neutron] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:27:41.960026 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cde36c-1fa5-4332-aa37-fdf9f236b2d2 service neutron] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:27:42.028843 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e154dd80-b0f7-458e-90d5-e285e901cd17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:27:42.029746 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e154dd80-b0f7-458e-90d5-e285e901cd17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:42.030028 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e154dd80-b0f7-458e-90d5-e285e901cd17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:42.030301 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e154dd80-b0f7-458e-90d5-e285e901cd17 admin admin] PATH_INFO: `/v3/projects/a70f42baac824e95be70eb103c352679` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:42.031619 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e154dd80-b0f7-458e-90d5-e285e901cd17 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a70f42baac824e95be70eb103c352679)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:27:42.033564 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e154dd80-b0f7-458e-90d5-e285e901cd17 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:27:42.127189 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:27:42.146320 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-e154dd80-b0f7-458e-90d5-e285e901cd17 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a70f42baac824e95be70eb103c352679', 'request_id': 'req-e154dd80-b0f7-458e-90d5-e285e901cd17'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:27:42.153685 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-e154dd80-b0f7-458e-90d5-e285e901cd17 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a70f42baac824e95be70eb103c352679', 'request_id': 'req-e154dd80-b0f7-458e-90d5-e285e901cd17'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:27:42.160648 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-e154dd80-b0f7-458e-90d5-e285e901cd17 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a70f42baac824e95be70eb103c352679', 'request_id': 'req-e154dd80-b0f7-458e-90d5-e285e901cd17'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:27:42.288371 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-e154dd80-b0f7-458e-90d5-e285e901cd17 admin admin] The token cache is being invalidate because project a70f42baac824e95be70eb103c352679 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 11:27:42.288669 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-e154dd80-b0f7-458e-90d5-e285e901cd17 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:27:42.289307 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-e154dd80-b0f7-458e-90d5-e285e901cd17 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:27:42.290965 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1181/2354] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jun 29 11:27:41 2022] DELETE /identity/v3/projects/a70f42baac824e95be70eb103c352679 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:27:42.296597 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fda1972-4640-4ccc-ae25-f29b6fc76ed6 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:42.296794 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fda1972-4640-4ccc-ae25-f29b6fc76ed6 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:42.297072 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fda1972-4640-4ccc-ae25-f29b6fc76ed6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:42.315940 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-4fda1972-4640-4ccc-ae25-f29b6fc76ed6 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:27:42.317163 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-4fda1972-4640-4ccc-ae25-f29b6fc76ed6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:27:42.414998 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1174/2355] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:27:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:27:42.423673 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04775578-52f2-44f0-a69b-56cfb1a36edb None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:42.423933 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04775578-52f2-44f0-a69b-56cfb1a36edb None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:42.424139 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04775578-52f2-44f0-a69b-56cfb1a36edb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:42.444529 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-04775578-52f2-44f0-a69b-56cfb1a36edb None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:27:42.445792 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-04775578-52f2-44f0-a69b-56cfb1a36edb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:27:42.464154 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1182/2356] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:27:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:27:42.472337 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fda1972-4640-4ccc-ae25-f29b6fc76ed6 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:27:42.491073 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fda1972-4640-4ccc-ae25-f29b6fc76ed6 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:27:42.498984 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6975d0d-c933-413d-878c-636d00a1991e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:27:42.499656 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6975d0d-c933-413d-878c-636d00a1991e admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:42.499973 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6975d0d-c933-413d-878c-636d00a1991e admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:42.500253 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6975d0d-c933-413d-878c-636d00a1991e admin admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:42.500990 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6975d0d-c933-413d-878c-636d00a1991e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:27:42.501298 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6975d0d-c933-413d-878c-636d00a1991e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:27:42.502903 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6975d0d-c933-413d-878c-636d00a1991e admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:27:42.521817 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1175/2357] 10.222.0.44 () {58 vars in 1152 bytes} [Wed Jun 29 11:27:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:27:42.530201 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04775578-52f2-44f0-a69b-56cfb1a36edb None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:27:42.539628 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04775578-52f2-44f0-a69b-56cfb1a36edb None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:27:42.547017 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eccae35-91dd-4be1-86ed-8f0bdf919524 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:27:42.547700 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eccae35-91dd-4be1-86ed-8f0bdf919524 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:42.548025 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eccae35-91dd-4be1-86ed-8f0bdf919524 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:42.548305 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eccae35-91dd-4be1-86ed-8f0bdf919524 admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:42.549015 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eccae35-91dd-4be1-86ed-8f0bdf919524 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:27:42.551051 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eccae35-91dd-4be1-86ed-8f0bdf919524 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:27:42.721705 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1183/2358] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jun 29 11:27:42 2022] POST /identity/v3/projects => generated 367 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:27:42.725115 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6975d0d-c933-413d-878c-636d00a1991e admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:27:42.737303 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6975d0d-c933-413d-878c-636d00a1991e admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:27:42.812660 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a507f8-3f5e-4391-ae74-36811d16cf8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:27:42.813546 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a507f8-3f5e-4391-ae74-36811d16cf8c admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:42.813904 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a507f8-3f5e-4391-ae74-36811d16cf8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:42.814292 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a507f8-3f5e-4391-ae74-36811d16cf8c admin admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:42.815126 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a507f8-3f5e-4391-ae74-36811d16cf8c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:27:42.818318 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a507f8-3f5e-4391-ae74-36811d16cf8c admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:27:42.912612 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1176/2359] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jun 29 11:27:42 2022] POST /identity/v3/users => generated 284 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:27:42.916543 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eccae35-91dd-4be1-86ed-8f0bdf919524 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:27:42.934931 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eccae35-91dd-4be1-86ed-8f0bdf919524 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:27:42.949012 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabaa5b1-0770-4253-8604-350c5088e9e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:27:42.949869 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabaa5b1-0770-4253-8604-350c5088e9e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:42.950096 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabaa5b1-0770-4253-8604-350c5088e9e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:42.950385 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabaa5b1-0770-4253-8604-350c5088e9e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:42.951133 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabaa5b1-0770-4253-8604-350c5088e9e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:27:42.958389 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabaa5b1-0770-4253-8604-350c5088e9e8 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:27:42.984655 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1184/2360] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:27:42 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 11:27:42.988356 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a507f8-3f5e-4391-ae74-36811d16cf8c admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:27:43.005130 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a507f8-3f5e-4391-ae74-36811d16cf8c admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:27:43.009834 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c4f477-bd8a-452a-a376-074f3e762f6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:27:43.010520 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c4f477-bd8a-452a-a376-074f3e762f6e admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:43.010805 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c4f477-bd8a-452a-a376-074f3e762f6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:43.011152 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c4f477-bd8a-452a-a376-074f3e762f6e admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:43.011942 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c4f477-bd8a-452a-a376-074f3e762f6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:27:43.013653 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c4f477-bd8a-452a-a376-074f3e762f6e admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:27:43.027187 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1177/2361] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:27:42 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 11:27:43.031525 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabaa5b1-0770-4253-8604-350c5088e9e8 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:27:43.046005 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabaa5b1-0770-4253-8604-350c5088e9e8 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:27:43.051422 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3860ee-ea31-41e6-a042-5213755565bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:27:43.052526 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3860ee-ea31-41e6-a042-5213755565bb admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:43.052922 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3860ee-ea31-41e6-a042-5213755565bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:43.053269 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3860ee-ea31-41e6-a042-5213755565bb admin admin] PATH_INFO: `/v3/projects/3eb664fdc4264ff783c647402adb0633/users/e2d913731d3d4c7f91741500dbe5892a/roles/e1c88eebe911432cab6a07fbeb84653b` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:43.073256 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3860ee-ea31-41e6-a042-5213755565bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=3eb664fdc4264ff783c647402adb0633, user_id=e2d913731d3d4c7f91741500dbe5892a, role_id=e1c88eebe911432cab6a07fbeb84653b)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:27:43.077211 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3860ee-ea31-41e6-a042-5213755565bb admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:27:43.188744 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1185/2362] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jun 29 11:27:43 2022] PUT /identity/v3/projects/3eb664fdc4264ff783c647402adb0633/users/e2d913731d3d4c7f91741500dbe5892a/roles/e1c88eebe911432cab6a07fbeb84653b => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 11:27:43.199571 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d06f212-7b4c-4a36-b6a0-d3db2c63499a None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:43.199925 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d06f212-7b4c-4a36-b6a0-d3db2c63499a None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:43.200214 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d06f212-7b4c-4a36-b6a0-d3db2c63499a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:43.227934 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-3d06f212-7b4c-4a36-b6a0-d3db2c63499a None None] MFA Rules not processed for user `e2d913731d3d4c7f91741500dbe5892a`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:27:43.229173 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-3d06f212-7b4c-4a36-b6a0-d3db2c63499a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:27:43.284365 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:27:43.310202 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1178/2363] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:27:43 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:27:43.322440 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3860ee-ea31-41e6-a042-5213755565bb admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:27:43.324374 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-ed3860ee-ea31-41e6-a042-5213755565bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:27:43.400920 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3860ee-ea31-41e6-a042-5213755565bb admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:27:43.404631 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ece9df7-cd6e-452b-8fe5-e63366537b6a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede3a17c7063432e8ee66462766584f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:27:43.405329 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ece9df7-cd6e-452b-8fe5-e63366537b6a service glance] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:43.405649 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ece9df7-cd6e-452b-8fe5-e63366537b6a service glance] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:43.405927 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ece9df7-cd6e-452b-8fe5-e63366537b6a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:43.415163 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ece9df7-cd6e-452b-8fe5-e63366537b6a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:27:43.416626 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ece9df7-cd6e-452b-8fe5-e63366537b6a service glance] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:27:43.425651 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1186/2364] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:27:43 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:27:47.275026 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aad7261-21aa-417e-b2d9-5c932e3a8dcf None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:47.275539 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aad7261-21aa-417e-b2d9-5c932e3a8dcf None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:47.275539 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aad7261-21aa-417e-b2d9-5c932e3a8dcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:47.308935 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-2aad7261-21aa-417e-b2d9-5c932e3a8dcf None None] MFA Rules not processed for user `e91cc0b93f484f759233469ec5f37bb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:27:47.312530 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-2aad7261-21aa-417e-b2d9-5c932e3a8dcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:27:47.399038 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1179/2365] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:27:47 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:27:47.400465 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ece9df7-cd6e-452b-8fe5-e63366537b6a service glance] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:27:47.404418 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-7ece9df7-cd6e-452b-8fe5-e63366537b6a service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:27:47.477923 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:27:47.484122 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ece9df7-cd6e-452b-8fe5-e63366537b6a service glance] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:27:47.488884 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39796605-70bf-475d-83b6-c8608cd17bc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:27:47.489553 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39796605-70bf-475d-83b6-c8608cd17bc2 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:47.489837 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39796605-70bf-475d-83b6-c8608cd17bc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:47.490852 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39796605-70bf-475d-83b6-c8608cd17bc2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:47.490852 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39796605-70bf-475d-83b6-c8608cd17bc2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:27:47.492796 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39796605-70bf-475d-83b6-c8608cd17bc2 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:27:47.582441 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1187/2366] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jun 29 11:27:47 2022] POST /identity/v3/projects => generated 379 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:27:47.586629 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aad7261-21aa-417e-b2d9-5c932e3a8dcf None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:27:47.598279 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aad7261-21aa-417e-b2d9-5c932e3a8dcf None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:27:47.659800 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993656c1-fd10-4464-b62b-f34758a3256a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:27:47.660605 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993656c1-fd10-4464-b62b-f34758a3256a admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:47.660891 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993656c1-fd10-4464-b62b-f34758a3256a admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:47.661171 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993656c1-fd10-4464-b62b-f34758a3256a admin admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:47.661890 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993656c1-fd10-4464-b62b-f34758a3256a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:27:47.663893 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993656c1-fd10-4464-b62b-f34758a3256a admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:27:47.775250 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1180/2367] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jun 29 11:27:47 2022] POST /identity/v3/users => generated 296 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:27:47.779292 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39796605-70bf-475d-83b6-c8608cd17bc2 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:27:47.790134 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39796605-70bf-475d-83b6-c8608cd17bc2 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:27:47.794800 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e05c44-38dc-4a2e-9486-e9445efaebda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:27:47.796096 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e05c44-38dc-4a2e-9486-e9445efaebda admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:47.796382 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e05c44-38dc-4a2e-9486-e9445efaebda admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:47.796657 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e05c44-38dc-4a2e-9486-e9445efaebda admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:47.797384 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e05c44-38dc-4a2e-9486-e9445efaebda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:27:47.798994 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e05c44-38dc-4a2e-9486-e9445efaebda admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:27:47.810591 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1188/2368] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:27:47 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 11:27:47.814897 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993656c1-fd10-4464-b62b-f34758a3256a admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:27:47.825621 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993656c1-fd10-4464-b62b-f34758a3256a admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:27:47.830420 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f406d2-36d8-4474-9312-ad24267ec21b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:27:47.831301 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f406d2-36d8-4474-9312-ad24267ec21b admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:47.831576 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f406d2-36d8-4474-9312-ad24267ec21b admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:47.831904 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f406d2-36d8-4474-9312-ad24267ec21b admin admin] PATH_INFO: `/v3/projects/7d0081550eb44279a4275e57d92f65e3/users/c49d176010fe49c38c7efc25926da2a3/roles/e95c2ce63e114cc19a9dda6ad21e794f` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:47.847816 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f406d2-36d8-4474-9312-ad24267ec21b admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d0081550eb44279a4275e57d92f65e3, user_id=c49d176010fe49c38c7efc25926da2a3, role_id=e95c2ce63e114cc19a9dda6ad21e794f)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:27:47.850509 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f406d2-36d8-4474-9312-ad24267ec21b admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:27:47.943103 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1181/2369] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jun 29 11:27:47 2022] PUT /identity/v3/projects/7d0081550eb44279a4275e57d92f65e3/users/c49d176010fe49c38c7efc25926da2a3/roles/e95c2ce63e114cc19a9dda6ad21e794f => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:27:47.946910 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e05c44-38dc-4a2e-9486-e9445efaebda admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:27:47.958826 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e05c44-38dc-4a2e-9486-e9445efaebda admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:27:48.021703 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb204262-4dd2-41e3-87a8-0d1fd1c4ec7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:27:48.022550 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb204262-4dd2-41e3-87a8-0d1fd1c4ec7b admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:48.022899 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb204262-4dd2-41e3-87a8-0d1fd1c4ec7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:48.023257 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb204262-4dd2-41e3-87a8-0d1fd1c4ec7b admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:48.024107 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb204262-4dd2-41e3-87a8-0d1fd1c4ec7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:27:48.025816 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb204262-4dd2-41e3-87a8-0d1fd1c4ec7b admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:27:48.036768 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1189/2370] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:27:47 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 11:27:48.040960 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f406d2-36d8-4474-9312-ad24267ec21b admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:27:48.052597 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f406d2-36d8-4474-9312-ad24267ec21b admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:27:48.057514 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96fd0c4-5082-431b-a24f-9191ddbe9d7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:27:48.058396 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96fd0c4-5082-431b-a24f-9191ddbe9d7e admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:48.058679 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96fd0c4-5082-431b-a24f-9191ddbe9d7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:48.058956 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96fd0c4-5082-431b-a24f-9191ddbe9d7e admin admin] PATH_INFO: `/v3/projects/7d0081550eb44279a4275e57d92f65e3/users/c49d176010fe49c38c7efc25926da2a3/roles/e95c2ce63e114cc19a9dda6ad21e794f` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:48.060965 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96fd0c4-5082-431b-a24f-9191ddbe9d7e admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d0081550eb44279a4275e57d92f65e3, user_id=c49d176010fe49c38c7efc25926da2a3, role_id=e95c2ce63e114cc19a9dda6ad21e794f)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:27:48.063634 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96fd0c4-5082-431b-a24f-9191ddbe9d7e admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:27:48.074206 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1182/2371] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jun 29 11:27:48 2022] PUT /identity/v3/projects/7d0081550eb44279a4275e57d92f65e3/users/c49d176010fe49c38c7efc25926da2a3/roles/e95c2ce63e114cc19a9dda6ad21e794f => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 11:27:50.038492 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb204262-4dd2-41e3-87a8-0d1fd1c4ec7b admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:27:50.040455 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-bb204262-4dd2-41e3-87a8-0d1fd1c4ec7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:27:50.204722 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb204262-4dd2-41e3-87a8-0d1fd1c4ec7b admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:27:50.284747 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:27:50.312344 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dce1a30-f27d-4553-badb-cdb4f25cdbdd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a0505342b704d02977a5125957c2c9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:27:50.313574 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dce1a30-f27d-4553-badb-cdb4f25cdbdd service placement] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:50.314195 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dce1a30-f27d-4553-badb-cdb4f25cdbdd service placement] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:50.314432 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dce1a30-f27d-4553-badb-cdb4f25cdbdd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:50.322806 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-4dce1a30-f27d-4553-badb-cdb4f25cdbdd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:27:50.365327 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dce1a30-f27d-4553-badb-cdb4f25cdbdd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:27:50.366957 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dce1a30-f27d-4553-badb-cdb4f25cdbdd service placement] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:27:50.397670 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1190/2372] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:27:50 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:27:56.110218 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96fd0c4-5082-431b-a24f-9191ddbe9d7e admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:27:56.113146 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-b96fd0c4-5082-431b-a24f-9191ddbe9d7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:27:56.155489 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:27:56.169656 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c5038b-04ae-43b0-952e-cfc2d5dc0f28 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:56.170749 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c5038b-04ae-43b0-952e-cfc2d5dc0f28 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:56.170749 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c5038b-04ae-43b0-952e-cfc2d5dc0f28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:56.221839 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-77c5038b-04ae-43b0-952e-cfc2d5dc0f28 None None] MFA Rules not processed for user `c49d176010fe49c38c7efc25926da2a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:27:56.223126 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-77c5038b-04ae-43b0-952e-cfc2d5dc0f28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:27:56.236322 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96fd0c4-5082-431b-a24f-9191ddbe9d7e admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:27:56.241725 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac98d634-21ff-422a-a6b8-fcddf11a1e2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:27:56.242696 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac98d634-21ff-422a-a6b8-fcddf11a1e2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:56.243211 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac98d634-21ff-422a-a6b8-fcddf11a1e2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:56.243692 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac98d634-21ff-422a-a6b8-fcddf11a1e2b admin admin] PATH_INFO: `/v3/users/d3f063fbca1f412fa65b888e09f0c59e` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:56.245949 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac98d634-21ff-422a-a6b8-fcddf11a1e2b admin admin] RBAC: Authorizing `identity:delete_user(user_id=d3f063fbca1f412fa65b888e09f0c59e)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:27:56.249320 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac98d634-21ff-422a-a6b8-fcddf11a1e2b admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:27:56.314890 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1191/2373] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:27:56 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:27:56.328941 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c5038b-04ae-43b0-952e-cfc2d5dc0f28 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:27:56.330327 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-77c5038b-04ae-43b0-952e-cfc2d5dc0f28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:27:56.445176 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c5038b-04ae-43b0-952e-cfc2d5dc0f28 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:27:56.450733 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9735a077-a794-4388-b6e2-4181f6354521 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '796aeeff16df4739bddec1f6dee61bfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:27:56.451415 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9735a077-a794-4388-b6e2-4181f6354521 service nova] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:56.451700 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9735a077-a794-4388-b6e2-4181f6354521 service nova] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:56.452007 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9735a077-a794-4388-b6e2-4181f6354521 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:56.486938 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:27:56.487622 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-ac98d634-21ff-422a-a6b8-fcddf11a1e2b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd3f063fbca1f412fa65b888e09f0c59e'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:27:56.510581 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9735a077-a794-4388-b6e2-4181f6354521 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:27:56.512246 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9735a077-a794-4388-b6e2-4181f6354521 service nova] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:27:56.521195 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1192/2374] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:27:56 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 11:27:56.555341 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-ac98d634-21ff-422a-a6b8-fcddf11a1e2b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd3f063fbca1f412fa65b888e09f0c59e'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:27:56.556307 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-ac98d634-21ff-422a-a6b8-fcddf11a1e2b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd3f063fbca1f412fa65b888e09f0c59e'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:27:56.568838 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-ac98d634-21ff-422a-a6b8-fcddf11a1e2b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd3f063fbca1f412fa65b888e09f0c59e'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:27:56.569499 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-ac98d634-21ff-422a-a6b8-fcddf11a1e2b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd3f063fbca1f412fa65b888e09f0c59e'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:27:56.582588 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1183/2375] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jun 29 11:27:56 2022] DELETE /identity/v3/users/d3f063fbca1f412fa65b888e09f0c59e => generated 0 bytes in 472 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:27:56.586037 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc96da0d-6c19-48fd-a718-c567dfcf427b None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:56.586311 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc96da0d-6c19-48fd-a718-c567dfcf427b None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:56.586591 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc96da0d-6c19-48fd-a718-c567dfcf427b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:56.604208 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-cc96da0d-6c19-48fd-a718-c567dfcf427b None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:27:56.605445 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-cc96da0d-6c19-48fd-a718-c567dfcf427b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:27:56.665560 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1193/2376] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:27:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:27:56.676545 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac98d634-21ff-422a-a6b8-fcddf11a1e2b admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:27:56.678481 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-ac98d634-21ff-422a-a6b8-fcddf11a1e2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:27:56.740608 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac98d634-21ff-422a-a6b8-fcddf11a1e2b admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:27:56.744445 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d80174-7e99-43a7-a884-bbe83c9f8d30 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f4bb9f989e4024a5da28dced646b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:27:56.745094 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d80174-7e99-43a7-a884-bbe83c9f8d30 service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:56.745414 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d80174-7e99-43a7-a884-bbe83c9f8d30 service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:56.745720 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d80174-7e99-43a7-a884-bbe83c9f8d30 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:56.755467 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d80174-7e99-43a7-a884-bbe83c9f8d30 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:27:56.757195 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d80174-7e99-43a7-a884-bbe83c9f8d30 service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:27:56.768099 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1184/2377] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:27:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:27:57.270149 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc96da0d-6c19-48fd-a718-c567dfcf427b None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:27:57.271649 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-cc96da0d-6c19-48fd-a718-c567dfcf427b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:27:57.285712 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc96da0d-6c19-48fd-a718-c567dfcf427b None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:27:57.290457 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f330a6-88aa-44f7-8c78-3fc22fe3cbed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:27:57.291327 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f330a6-88aa-44f7-8c78-3fc22fe3cbed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:57.291609 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f330a6-88aa-44f7-8c78-3fc22fe3cbed admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:57.292056 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f330a6-88aa-44f7-8c78-3fc22fe3cbed admin admin] PATH_INFO: `/v3/projects/917389e0ce224a0a8aaab3fc826a0214` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:57.293374 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f330a6-88aa-44f7-8c78-3fc22fe3cbed admin admin] RBAC: Authorizing `identity:delete_project(project_id=917389e0ce224a0a8aaab3fc826a0214)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:27:57.295487 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f330a6-88aa-44f7-8c78-3fc22fe3cbed admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:27:57.415796 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-73f330a6-88aa-44f7-8c78-3fc22fe3cbed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '917389e0ce224a0a8aaab3fc826a0214', 'request_id': 'req-73f330a6-88aa-44f7-8c78-3fc22fe3cbed'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:27:57.423052 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-73f330a6-88aa-44f7-8c78-3fc22fe3cbed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '917389e0ce224a0a8aaab3fc826a0214', 'request_id': 'req-73f330a6-88aa-44f7-8c78-3fc22fe3cbed'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:27:57.429037 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-73f330a6-88aa-44f7-8c78-3fc22fe3cbed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '917389e0ce224a0a8aaab3fc826a0214', 'request_id': 'req-73f330a6-88aa-44f7-8c78-3fc22fe3cbed'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:27:57.531226 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-73f330a6-88aa-44f7-8c78-3fc22fe3cbed admin admin] The token cache is being invalidate because project 917389e0ce224a0a8aaab3fc826a0214 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 11:27:57.531538 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-73f330a6-88aa-44f7-8c78-3fc22fe3cbed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:27:57.532218 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-73f330a6-88aa-44f7-8c78-3fc22fe3cbed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:27:57.542428 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1194/2378] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jun 29 11:27:57 2022] DELETE /identity/v3/projects/917389e0ce224a0a8aaab3fc826a0214 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:27:57.542564 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48514b48-b997-4d76-b0c3-6d367bff8a61 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:57.542564 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48514b48-b997-4d76-b0c3-6d367bff8a61 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:57.542564 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48514b48-b997-4d76-b0c3-6d367bff8a61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:57.620906 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-48514b48-b997-4d76-b0c3-6d367bff8a61 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:27:57.622120 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-48514b48-b997-4d76-b0c3-6d367bff8a61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:27:57.675772 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:27:57.687928 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1185/2379] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:27:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:27:57.695538 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20203eb-d5b4-4301-92f6-be5537e93f29 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:57.695855 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20203eb-d5b4-4301-92f6-be5537e93f29 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:57.696153 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20203eb-d5b4-4301-92f6-be5537e93f29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:57.715404 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-c20203eb-d5b4-4301-92f6-be5537e93f29 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:27:57.716762 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-c20203eb-d5b4-4301-92f6-be5537e93f29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:27:57.725112 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1195/2380] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:27:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:27:57.729434 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48514b48-b997-4d76-b0c3-6d367bff8a61 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:27:57.739787 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48514b48-b997-4d76-b0c3-6d367bff8a61 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:27:57.744450 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaad3067-41e0-45b1-8286-aa5088699127 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:27:57.745109 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaad3067-41e0-45b1-8286-aa5088699127 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:57.745394 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaad3067-41e0-45b1-8286-aa5088699127 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:57.745668 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaad3067-41e0-45b1-8286-aa5088699127 admin admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:57.746405 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaad3067-41e0-45b1-8286-aa5088699127 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:27:57.746712 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaad3067-41e0-45b1-8286-aa5088699127 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:27:57.748964 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaad3067-41e0-45b1-8286-aa5088699127 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:27:57.762412 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1186/2381] 10.222.0.44 () {58 vars in 1152 bytes} [Wed Jun 29 11:27:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:27:57.765412 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20203eb-d5b4-4301-92f6-be5537e93f29 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:27:57.775474 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20203eb-d5b4-4301-92f6-be5537e93f29 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:27:57.782076 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e3d895-1ace-49c8-9f2d-b96fb97ffd25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:27:57.782076 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e3d895-1ace-49c8-9f2d-b96fb97ffd25 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:57.782330 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e3d895-1ace-49c8-9f2d-b96fb97ffd25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:57.782610 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e3d895-1ace-49c8-9f2d-b96fb97ffd25 admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:57.783353 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e3d895-1ace-49c8-9f2d-b96fb97ffd25 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:27:57.785354 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e3d895-1ace-49c8-9f2d-b96fb97ffd25 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:27:57.888157 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1196/2382] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jun 29 11:27:57 2022] POST /identity/v3/projects => generated 383 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:27:57.890004 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaad3067-41e0-45b1-8286-aa5088699127 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:27:57.903922 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaad3067-41e0-45b1-8286-aa5088699127 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:27:57.963469 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c827d444-a957-41cb-99ea-4ccddbced671 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:27:57.964200 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c827d444-a957-41cb-99ea-4ccddbced671 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:57.964484 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c827d444-a957-41cb-99ea-4ccddbced671 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:57.964761 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c827d444-a957-41cb-99ea-4ccddbced671 admin admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:57.965459 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c827d444-a957-41cb-99ea-4ccddbced671 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:27:57.967939 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c827d444-a957-41cb-99ea-4ccddbced671 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:27:58.057689 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1187/2383] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jun 29 11:27:57 2022] POST /identity/v3/users => generated 292 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:27:58.058880 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e3d895-1ace-49c8-9f2d-b96fb97ffd25 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:27:58.070363 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e3d895-1ace-49c8-9f2d-b96fb97ffd25 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:27:58.076382 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa761003-1190-4dee-8146-8ebbb7d382ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:27:58.077070 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa761003-1190-4dee-8146-8ebbb7d382ce admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:58.077357 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa761003-1190-4dee-8146-8ebbb7d382ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:58.077798 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa761003-1190-4dee-8146-8ebbb7d382ce admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:58.078713 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa761003-1190-4dee-8146-8ebbb7d382ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:27:58.080200 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa761003-1190-4dee-8146-8ebbb7d382ce admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:27:58.094398 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1197/2384] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:27:58 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 11:27:58.098544 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c827d444-a957-41cb-99ea-4ccddbced671 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:27:58.109681 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c827d444-a957-41cb-99ea-4ccddbced671 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:27:58.141814 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046cc5b9-4d45-4ad8-81d3-d7e7e1bd75af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:27:58.142571 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046cc5b9-4d45-4ad8-81d3-d7e7e1bd75af admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:58.142920 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046cc5b9-4d45-4ad8-81d3-d7e7e1bd75af admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:58.143274 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046cc5b9-4d45-4ad8-81d3-d7e7e1bd75af admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:58.144146 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046cc5b9-4d45-4ad8-81d3-d7e7e1bd75af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:27:58.145953 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046cc5b9-4d45-4ad8-81d3-d7e7e1bd75af admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:27:58.162023 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1188/2385] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:27:58 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 11:27:58.166298 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa761003-1190-4dee-8146-8ebbb7d382ce admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:27:58.177074 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa761003-1190-4dee-8146-8ebbb7d382ce admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:27:58.182273 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dcacb3-5121-4d99-b4ce-4944336e6f8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:27:58.183249 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dcacb3-5121-4d99-b4ce-4944336e6f8d admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:58.183607 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dcacb3-5121-4d99-b4ce-4944336e6f8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:58.184063 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dcacb3-5121-4d99-b4ce-4944336e6f8d admin admin] PATH_INFO: `/v3/projects/58aac09bb66b4f98abf47ae8bd863203/users/1158e0283f3643669448ca4281e7b1fa/roles/e1c88eebe911432cab6a07fbeb84653b` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:58.200889 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57dcacb3-5121-4d99-b4ce-4944336e6f8d admin admin] RBAC: Authorizing `identity:create_grant(project_id=58aac09bb66b4f98abf47ae8bd863203, user_id=1158e0283f3643669448ca4281e7b1fa, role_id=e1c88eebe911432cab6a07fbeb84653b)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:27:58.205264 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57dcacb3-5121-4d99-b4ce-4944336e6f8d admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:27:58.207802 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:27:58.307569 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1198/2386] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jun 29 11:27:58 2022] PUT /identity/v3/projects/58aac09bb66b4f98abf47ae8bd863203/users/1158e0283f3643669448ca4281e7b1fa/roles/e1c88eebe911432cab6a07fbeb84653b => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:27:58.312755 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca7543e-a829-43e3-b685-a625f326e8fe None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:58.313155 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca7543e-a829-43e3-b685-a625f326e8fe None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:58.313432 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca7543e-a829-43e3-b685-a625f326e8fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:58.331876 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-cca7543e-a829-43e3-b685-a625f326e8fe None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:27:58.333128 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-cca7543e-a829-43e3-b685-a625f326e8fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:27:58.407883 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1189/2387] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:27:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:27:58.418221 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dcacb3-5121-4d99-b4ce-4944336e6f8d admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:27:58.420318 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-57dcacb3-5121-4d99-b4ce-4944336e6f8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:27:58.500719 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dcacb3-5121-4d99-b4ce-4944336e6f8d admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:27:58.504660 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a549e6d-4ff3-4232-b5eb-fcda2bf47639 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f4bb9f989e4024a5da28dced646b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:27:58.505334 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a549e6d-4ff3-4232-b5eb-fcda2bf47639 service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:27:58.505702 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a549e6d-4ff3-4232-b5eb-fcda2bf47639 service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:27:58.506074 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a549e6d-4ff3-4232-b5eb-fcda2bf47639 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:27:58.518062 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a549e6d-4ff3-4232-b5eb-fcda2bf47639 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:27:58.519788 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a549e6d-4ff3-4232-b5eb-fcda2bf47639 service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:27:58.531157 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1199/2388] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:27:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:28:04.150249 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a47939f-f76f-4c0d-b971-fce4bd866429 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:28:04.151710 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a47939f-f76f-4c0d-b971-fce4bd866429 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:28:04.152547 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a47939f-f76f-4c0d-b971-fce4bd866429 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:28:04.239646 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-9a47939f-f76f-4c0d-b971-fce4bd866429 None None] MFA Rules not processed for user `55487c2107a44c15b656b4865ee0d4fa`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:28:04.241580 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-9a47939f-f76f-4c0d-b971-fce4bd866429 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:28:04.287079 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:28:04.353237 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1190/2389] 10.222.0.44 () {58 vars in 1009 bytes} [Wed Jun 29 11:28:04 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:28:04.423890 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a549e6d-4ff3-4232-b5eb-fcda2bf47639 service neutron] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:28:04.423890 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-1a549e6d-4ff3-4232-b5eb-fcda2bf47639 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:28:04.551197 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a549e6d-4ff3-4232-b5eb-fcda2bf47639 service neutron] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:28:04.557595 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b81219-e513-46db-931f-5b6fee1156fd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3b38493ba904759badc800e396befda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:28:04.558271 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b81219-e513-46db-931f-5b6fee1156fd service swift] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:28:04.558968 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b81219-e513-46db-931f-5b6fee1156fd service swift] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:28:04.558968 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b81219-e513-46db-931f-5b6fee1156fd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:28:04.571030 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b81219-e513-46db-931f-5b6fee1156fd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:28:04.571653 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b81219-e513-46db-931f-5b6fee1156fd service swift] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:28:04.588128 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1200/2390] 10.222.0.44 () {60 vars in 1384 bytes} [Wed Jun 29 11:28:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 29 11:28:05.229692 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5855127f-6f91-4a23-9952-0b6a886f45af None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:28:05.230207 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5855127f-6f91-4a23-9952-0b6a886f45af None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:28:05.230326 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5855127f-6f91-4a23-9952-0b6a886f45af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:28:05.260782 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-5855127f-6f91-4a23-9952-0b6a886f45af None None] MFA Rules not processed for user `1158e0283f3643669448ca4281e7b1fa`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:28:05.262014 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-5855127f-6f91-4a23-9952-0b6a886f45af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:28:05.350175 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1191/2391] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:28:05 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 11:28:05.354416 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b81219-e513-46db-931f-5b6fee1156fd service swift] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:28:05.365891 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b81219-e513-46db-931f-5b6fee1156fd service swift] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:28:05.370479 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe85d4d-fb1c-4c47-a5a9-2ab62d954d84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:28:05.371110 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe85d4d-fb1c-4c47-a5a9-2ab62d954d84 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:28:05.371403 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe85d4d-fb1c-4c47-a5a9-2ab62d954d84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:28:05.371679 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe85d4d-fb1c-4c47-a5a9-2ab62d954d84 admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:28:05.372437 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe85d4d-fb1c-4c47-a5a9-2ab62d954d84 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:28:05.376010 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe85d4d-fb1c-4c47-a5a9-2ab62d954d84 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:28:05.498038 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1201/2392] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jun 29 11:28:05 2022] POST /identity/v3/projects => generated 385 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:28:05.504436 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5855127f-6f91-4a23-9952-0b6a886f45af None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:28:05.514273 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5855127f-6f91-4a23-9952-0b6a886f45af None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:28:05.584359 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb409e3-430a-4d28-b31c-8a0f8a821dff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:28:05.585056 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb409e3-430a-4d28-b31c-8a0f8a821dff admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:28:05.585318 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb409e3-430a-4d28-b31c-8a0f8a821dff admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:28:05.585594 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb409e3-430a-4d28-b31c-8a0f8a821dff admin admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:28:05.586300 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb409e3-430a-4d28-b31c-8a0f8a821dff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:28:05.588288 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb409e3-430a-4d28-b31c-8a0f8a821dff admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:28:05.665909 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1418ad-46db-41bc-b6b5-0a41f180f370 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:28:05.666373 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1418ad-46db-41bc-b6b5-0a41f180f370 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:28:05.666812 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1418ad-46db-41bc-b6b5-0a41f180f370 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:28:05.687936 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-1e1418ad-46db-41bc-b6b5-0a41f180f370 None None] MFA Rules not processed for user `55487c2107a44c15b656b4865ee0d4fa`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:28:05.689220 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-1e1418ad-46db-41bc-b6b5-0a41f180f370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:28:05.703547 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1192/2393] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jun 29 11:28:05 2022] POST /identity/v3/users => generated 299 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 11:28:05.707576 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb409e3-430a-4d28-b31c-8a0f8a821dff admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:28:05.719558 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb409e3-430a-4d28-b31c-8a0f8a821dff admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:28:05.724895 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1158d9a3-7ade-4602-9d97-0bfbca7e56fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:28:05.725596 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1158d9a3-7ade-4602-9d97-0bfbca7e56fb admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:28:05.725984 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1158d9a3-7ade-4602-9d97-0bfbca7e56fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:28:05.726362 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1158d9a3-7ade-4602-9d97-0bfbca7e56fb admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:28:05.727196 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1158d9a3-7ade-4602-9d97-0bfbca7e56fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:28:05.728962 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1158d9a3-7ade-4602-9d97-0bfbca7e56fb admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:28:05.740654 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1193/2394] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:28:05 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 11:28:05.744616 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1158d9a3-7ade-4602-9d97-0bfbca7e56fb admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:28:05.747552 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:28:05.755202 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1158d9a3-7ade-4602-9d97-0bfbca7e56fb admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:28:05.760067 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59336a40-6758-46e9-b79a-3ca5e85b43bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:28:05.761017 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59336a40-6758-46e9-b79a-3ca5e85b43bf admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:28:05.761301 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59336a40-6758-46e9-b79a-3ca5e85b43bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:28:05.761580 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59336a40-6758-46e9-b79a-3ca5e85b43bf admin admin] PATH_INFO: `/v3/projects/d74b3636742f4d8aa46082cf1b0875f2/users/82eaaa92763d40568351ef330bd68111/roles/e95c2ce63e114cc19a9dda6ad21e794f` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:28:05.763154 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1202/2395] 10.222.0.44 () {58 vars in 967 bytes} [Wed Jun 29 11:28:05 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 11:28:05.776453 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1418ad-46db-41bc-b6b5-0a41f180f370 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:28:05.783328 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59336a40-6758-46e9-b79a-3ca5e85b43bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=d74b3636742f4d8aa46082cf1b0875f2, user_id=82eaaa92763d40568351ef330bd68111, role_id=e95c2ce63e114cc19a9dda6ad21e794f)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:28:05.786189 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59336a40-6758-46e9-b79a-3ca5e85b43bf admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:28:05.845703 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1194/2396] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jun 29 11:28:05 2022] PUT /identity/v3/projects/d74b3636742f4d8aa46082cf1b0875f2/users/82eaaa92763d40568351ef330bd68111/roles/e95c2ce63e114cc19a9dda6ad21e794f => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:28:05.846370 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1418ad-46db-41bc-b6b5-0a41f180f370 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:28:05.849472 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59336a40-6758-46e9-b79a-3ca5e85b43bf admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:28:05.860102 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:28:05.921194 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59336a40-6758-46e9-b79a-3ca5e85b43bf admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:28:05.926625 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7e2e00-cb87-452c-a0f1-33e2dfaa8757 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3b38493ba904759badc800e396befda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:28:05.927675 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7e2e00-cb87-452c-a0f1-33e2dfaa8757 service swift] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:28:05.927675 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7e2e00-cb87-452c-a0f1-33e2dfaa8757 service swift] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:28:05.927914 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7e2e00-cb87-452c-a0f1-33e2dfaa8757 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:28:05.930646 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcf9d3b-4661-427e-8cad-2cd0a85b6c6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:28:05.931316 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcf9d3b-4661-427e-8cad-2cd0a85b6c6f admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:28:05.931594 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcf9d3b-4661-427e-8cad-2cd0a85b6c6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:28:05.931891 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcf9d3b-4661-427e-8cad-2cd0a85b6c6f admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:28:05.932664 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcf9d3b-4661-427e-8cad-2cd0a85b6c6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:28:05.934301 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcf9d3b-4661-427e-8cad-2cd0a85b6c6f admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:28:05.946145 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1195/2397] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:28:05 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 11:28:05.954975 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcf9d3b-4661-427e-8cad-2cd0a85b6c6f admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:28:05.964938 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcf9d3b-4661-427e-8cad-2cd0a85b6c6f admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:28:05.970232 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13ba33e-1dae-4e6d-a034-61a0084762e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:28:05.971377 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13ba33e-1dae-4e6d-a034-61a0084762e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:28:05.971656 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13ba33e-1dae-4e6d-a034-61a0084762e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:28:05.971972 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13ba33e-1dae-4e6d-a034-61a0084762e7 admin admin] PATH_INFO: `/v3/projects/d74b3636742f4d8aa46082cf1b0875f2/users/82eaaa92763d40568351ef330bd68111/roles/e95c2ce63e114cc19a9dda6ad21e794f` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:28:05.974414 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13ba33e-1dae-4e6d-a034-61a0084762e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d74b3636742f4d8aa46082cf1b0875f2, user_id=82eaaa92763d40568351ef330bd68111, role_id=e95c2ce63e114cc19a9dda6ad21e794f)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:28:05.977856 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13ba33e-1dae-4e6d-a034-61a0084762e7 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:28:05.988920 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1196/2398] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jun 29 11:28:05 2022] PUT /identity/v3/projects/d74b3636742f4d8aa46082cf1b0875f2/users/82eaaa92763d40568351ef330bd68111/roles/e95c2ce63e114cc19a9dda6ad21e794f => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:28:05.991085 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7e2e00-cb87-452c-a0f1-33e2dfaa8757 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:28:05.993106 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7e2e00-cb87-452c-a0f1-33e2dfaa8757 service swift] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:28:06.020652 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1203/2399] 10.222.0.44 () {60 vars in 1384 bytes} [Wed Jun 29 11:28:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 29 11:28:07.038104 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c58802-e4e8-4210-b380-668cab3a8f53 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:28:07.038613 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c58802-e4e8-4210-b380-668cab3a8f53 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:28:07.038613 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c58802-e4e8-4210-b380-668cab3a8f53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:28:07.062467 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-78c58802-e4e8-4210-b380-668cab3a8f53 None None] MFA Rules not processed for user `55487c2107a44c15b656b4865ee0d4fa`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:28:07.063894 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-78c58802-e4e8-4210-b380-668cab3a8f53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:28:07.136492 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1197/2400] 10.222.0.44 () {58 vars in 1009 bytes} [Wed Jun 29 11:28:07 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:28:07.152155 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7e2e00-cb87-452c-a0f1-33e2dfaa8757 service swift] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:28:07.167703 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7e2e00-cb87-452c-a0f1-33e2dfaa8757 service swift] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:28:07.216276 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:28:07.249863 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1cb701-711f-426a-bce6-620b67c20ac9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3b38493ba904759badc800e396befda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:28:07.250538 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1cb701-711f-426a-bce6-620b67c20ac9 service swift] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:28:07.250823 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1cb701-711f-426a-bce6-620b67c20ac9 service swift] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:28:07.251272 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1cb701-711f-426a-bce6-620b67c20ac9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:28:07.267725 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1cb701-711f-426a-bce6-620b67c20ac9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:28:07.267725 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1cb701-711f-426a-bce6-620b67c20ac9 service swift] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:28:07.273853 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1204/2401] 10.222.0.44 () {60 vars in 1384 bytes} [Wed Jun 29 11:28:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 29 11:28:07.964084 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418b3f39-029d-4604-9e60-9d45d29ad753 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:28:07.965200 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418b3f39-029d-4604-9e60-9d45d29ad753 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:28:07.965200 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418b3f39-029d-4604-9e60-9d45d29ad753 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:28:07.983616 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-418b3f39-029d-4604-9e60-9d45d29ad753 None None] MFA Rules not processed for user `55487c2107a44c15b656b4865ee0d4fa`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:28:07.984861 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-418b3f39-029d-4604-9e60-9d45d29ad753 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:28:07.992656 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1198/2402] 10.222.0.44 () {58 vars in 967 bytes} [Wed Jun 29 11:28:07 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:28:08.006920 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1cb701-711f-426a-bce6-620b67c20ac9 service swift] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:28:08.020150 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1cb701-711f-426a-bce6-620b67c20ac9 service swift] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:28:08.029380 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f6abd7-fa46-4737-887a-23cec07d2e67 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3b38493ba904759badc800e396befda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:28:08.030358 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f6abd7-fa46-4737-887a-23cec07d2e67 service swift] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:28:08.030887 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f6abd7-fa46-4737-887a-23cec07d2e67 service swift] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:28:08.031384 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f6abd7-fa46-4737-887a-23cec07d2e67 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:28:08.044309 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f6abd7-fa46-4737-887a-23cec07d2e67 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:28:08.047029 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f6abd7-fa46-4737-887a-23cec07d2e67 service swift] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:28:08.058594 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1205/2403] 10.222.0.44 () {60 vars in 1384 bytes} [Wed Jun 29 11:28:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 29 11:28:11.192726 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418b3f39-029d-4604-9e60-9d45d29ad753 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:28:11.194627 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-418b3f39-029d-4604-9e60-9d45d29ad753 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:28:11.281302 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418b3f39-029d-4604-9e60-9d45d29ad753 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:28:11.286452 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2754893-acc0-43d7-b598-a5243ac870cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:28:11.287405 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2754893-acc0-43d7-b598-a5243ac870cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:28:11.287913 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2754893-acc0-43d7-b598-a5243ac870cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:28:11.288256 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2754893-acc0-43d7-b598-a5243ac870cd admin admin] PATH_INFO: `/v3/users/907e9be3b1f54f17b8e3841130d41cc7` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:28:11.289619 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2754893-acc0-43d7-b598-a5243ac870cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=907e9be3b1f54f17b8e3841130d41cc7)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:28:11.291599 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2754893-acc0-43d7-b598-a5243ac870cd admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:28:11.453677 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:28:11.479354 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-d2754893-acc0-43d7-b598-a5243ac870cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '907e9be3b1f54f17b8e3841130d41cc7'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:28:11.576886 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-d2754893-acc0-43d7-b598-a5243ac870cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '907e9be3b1f54f17b8e3841130d41cc7'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:28:11.577610 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-d2754893-acc0-43d7-b598-a5243ac870cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '907e9be3b1f54f17b8e3841130d41cc7'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:28:11.591284 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-d2754893-acc0-43d7-b598-a5243ac870cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '907e9be3b1f54f17b8e3841130d41cc7'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:28:11.592222 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-d2754893-acc0-43d7-b598-a5243ac870cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '907e9be3b1f54f17b8e3841130d41cc7'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:28:11.610797 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1199/2404] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jun 29 11:28:11 2022] DELETE /identity/v3/users/907e9be3b1f54f17b8e3841130d41cc7 => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:28:11.615502 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882e8bf7-1e21-4957-9352-26eac9ec2c8a None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:28:11.616162 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882e8bf7-1e21-4957-9352-26eac9ec2c8a None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:28:11.616162 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882e8bf7-1e21-4957-9352-26eac9ec2c8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:28:11.635494 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-882e8bf7-1e21-4957-9352-26eac9ec2c8a None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:28:11.636822 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-882e8bf7-1e21-4957-9352-26eac9ec2c8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:28:11.717193 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1206/2405] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:28:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:28:11.728884 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2754893-acc0-43d7-b598-a5243ac870cd admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:28:11.730491 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-d2754893-acc0-43d7-b598-a5243ac870cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:28:11.812136 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2754893-acc0-43d7-b598-a5243ac870cd admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:28:11.817016 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2c2b12-8247-4931-a381-92ec5ac42fc4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f4bb9f989e4024a5da28dced646b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:28:11.817927 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2c2b12-8247-4931-a381-92ec5ac42fc4 service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:28:11.818209 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2c2b12-8247-4931-a381-92ec5ac42fc4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:28:11.818499 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2c2b12-8247-4931-a381-92ec5ac42fc4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:28:11.830847 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2c2b12-8247-4931-a381-92ec5ac42fc4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:28:11.835992 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2c2b12-8247-4931-a381-92ec5ac42fc4 service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:28:11.851588 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1200/2406] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:28:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 11:28:12.271235 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882e8bf7-1e21-4957-9352-26eac9ec2c8a None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:28:12.281368 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-882e8bf7-1e21-4957-9352-26eac9ec2c8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:28:12.310510 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882e8bf7-1e21-4957-9352-26eac9ec2c8a None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:28:12.315764 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7012e734-e8cc-457c-bde2-bfd3518bdacf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:28:12.321179 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7012e734-e8cc-457c-bde2-bfd3518bdacf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:28:12.321734 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7012e734-e8cc-457c-bde2-bfd3518bdacf admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:28:12.322223 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7012e734-e8cc-457c-bde2-bfd3518bdacf admin admin] PATH_INFO: `/v3/projects/c7653e362c7d49a3ae386f137c41889c` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:28:12.324232 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7012e734-e8cc-457c-bde2-bfd3518bdacf admin admin] RBAC: Authorizing `identity:delete_project(project_id=c7653e362c7d49a3ae386f137c41889c)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:28:12.326586 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7012e734-e8cc-457c-bde2-bfd3518bdacf admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:28:12.461256 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-7012e734-e8cc-457c-bde2-bfd3518bdacf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c7653e362c7d49a3ae386f137c41889c', 'request_id': 'req-7012e734-e8cc-457c-bde2-bfd3518bdacf'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:28:12.477168 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-7012e734-e8cc-457c-bde2-bfd3518bdacf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c7653e362c7d49a3ae386f137c41889c', 'request_id': 'req-7012e734-e8cc-457c-bde2-bfd3518bdacf'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:28:12.487358 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-7012e734-e8cc-457c-bde2-bfd3518bdacf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c7653e362c7d49a3ae386f137c41889c', 'request_id': 'req-7012e734-e8cc-457c-bde2-bfd3518bdacf'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:28:12.592122 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-7012e734-e8cc-457c-bde2-bfd3518bdacf admin admin] The token cache is being invalidate because project c7653e362c7d49a3ae386f137c41889c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 11:28:12.593987 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-7012e734-e8cc-457c-bde2-bfd3518bdacf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:28:12.608549 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-7012e734-e8cc-457c-bde2-bfd3518bdacf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:28:12.623871 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1207/2407] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jun 29 11:28:12 2022] DELETE /identity/v3/projects/c7653e362c7d49a3ae386f137c41889c => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:28:12.666194 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61a04d3-e771-48c6-8ada-2a52d815ef34 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:28:12.667282 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61a04d3-e771-48c6-8ada-2a52d815ef34 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:28:12.667677 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61a04d3-e771-48c6-8ada-2a52d815ef34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:28:12.723152 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-c61a04d3-e771-48c6-8ada-2a52d815ef34 None None] MFA Rules not processed for user `82eaaa92763d40568351ef330bd68111`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:28:12.724800 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-c61a04d3-e771-48c6-8ada-2a52d815ef34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:28:12.835864 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1201/2408] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:28:12 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:28:12.838475 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7012e734-e8cc-457c-bde2-bfd3518bdacf admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:28:12.840121 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-7012e734-e8cc-457c-bde2-bfd3518bdacf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:28:12.883455 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:28:12.961831 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7012e734-e8cc-457c-bde2-bfd3518bdacf admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:28:12.966683 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d937770-9b19-4646-b8a1-04bf7a5f60d2 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede3a17c7063432e8ee66462766584f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:28:12.967432 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d937770-9b19-4646-b8a1-04bf7a5f60d2 service glance] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:28:12.967777 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d937770-9b19-4646-b8a1-04bf7a5f60d2 service glance] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:28:12.968327 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d937770-9b19-4646-b8a1-04bf7a5f60d2 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:28:12.970622 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-3d937770-9b19-4646-b8a1-04bf7a5f60d2 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:28:13.043375 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d937770-9b19-4646-b8a1-04bf7a5f60d2 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:28:13.045182 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d937770-9b19-4646-b8a1-04bf7a5f60d2 service glance] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:28:13.055322 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1208/2409] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:28:12 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 11:28:13.338878 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cb71a5-eda3-4332-8dec-2e22f4925c2a None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:28:13.339364 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cb71a5-eda3-4332-8dec-2e22f4925c2a None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:28:13.339789 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cb71a5-eda3-4332-8dec-2e22f4925c2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:28:13.359378 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-24cb71a5-eda3-4332-8dec-2e22f4925c2a None None] MFA Rules not processed for user `55487c2107a44c15b656b4865ee0d4fa`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:28:13.360663 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-24cb71a5-eda3-4332-8dec-2e22f4925c2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:28:13.399698 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:28:13.422499 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1202/2410] 10.222.0.44 () {58 vars in 1009 bytes} [Wed Jun 29 11:28:13 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:28:13.436022 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d937770-9b19-4646-b8a1-04bf7a5f60d2 service glance] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:28:13.437451 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-3d937770-9b19-4646-b8a1-04bf7a5f60d2 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:28:13.508428 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d937770-9b19-4646-b8a1-04bf7a5f60d2 service glance] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:28:13.514355 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8964849d-d9ce-4aae-be08-96de97c75e0a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3b38493ba904759badc800e396befda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:28:13.515366 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8964849d-d9ce-4aae-be08-96de97c75e0a service swift] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:28:13.515662 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8964849d-d9ce-4aae-be08-96de97c75e0a service swift] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:28:13.516042 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8964849d-d9ce-4aae-be08-96de97c75e0a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:28:13.524858 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8964849d-d9ce-4aae-be08-96de97c75e0a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:28:13.526463 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8964849d-d9ce-4aae-be08-96de97c75e0a service swift] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:28:13.535480 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1209/2411] 10.222.0.44 () {60 vars in 1384 bytes} [Wed Jun 29 11:28:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 29 11:28:23.518769 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=8 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:28:23.531574 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=74 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:28:24.184240 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a59b26-8d36-4278-8a92-e10c107349b4 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:28:24.184643 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a59b26-8d36-4278-8a92-e10c107349b4 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:28:24.184933 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a59b26-8d36-4278-8a92-e10c107349b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:28:24.202543 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-d6a59b26-8d36-4278-8a92-e10c107349b4 None None] MFA Rules not processed for user `1158e0283f3643669448ca4281e7b1fa`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:28:24.203777 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-d6a59b26-8d36-4278-8a92-e10c107349b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:28:24.210728 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1203/2412] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:28:24 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:28:29.985753 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bcf753-4747-4fbd-a983-7db9c81c2db7 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:28:29.986853 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bcf753-4747-4fbd-a983-7db9c81c2db7 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:28:29.987238 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bcf753-4747-4fbd-a983-7db9c81c2db7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:28:30.006029 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-c9bcf753-4747-4fbd-a983-7db9c81c2db7 None None] MFA Rules not processed for user `55487c2107a44c15b656b4865ee0d4fa`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:28:30.007407 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-c9bcf753-4747-4fbd-a983-7db9c81c2db7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:28:30.018377 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1210/2413] 10.222.0.44 () {58 vars in 1009 bytes} [Wed Jun 29 11:28:29 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:28:30.033590 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a59b26-8d36-4278-8a92-e10c107349b4 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:28:30.047468 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a59b26-8d36-4278-8a92-e10c107349b4 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:28:30.052687 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa45f0b-1767-48fa-91c7-6804024d6766 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3b38493ba904759badc800e396befda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:28:30.053153 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa45f0b-1767-48fa-91c7-6804024d6766 service swift] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:28:30.053728 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa45f0b-1767-48fa-91c7-6804024d6766 service swift] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:28:30.053728 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa45f0b-1767-48fa-91c7-6804024d6766 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:28:30.062445 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa45f0b-1767-48fa-91c7-6804024d6766 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:28:30.064308 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa45f0b-1767-48fa-91c7-6804024d6766 service swift] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:28:30.072733 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1204/2414] 10.222.0.44 () {60 vars in 1384 bytes} [Wed Jun 29 11:28:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 29 11:28:37.960739 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f2b61b-0cca-46e2-a542-f94ebda28948 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:28:37.961371 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f2b61b-0cca-46e2-a542-f94ebda28948 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:28:37.961371 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f2b61b-0cca-46e2-a542-f94ebda28948 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:28:37.980310 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-86f2b61b-0cca-46e2-a542-f94ebda28948 None None] MFA Rules not processed for user `e91cc0b93f484f759233469ec5f37bb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:28:37.987494 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-86f2b61b-0cca-46e2-a542-f94ebda28948 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:28:38.047805 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1211/2415] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:28:37 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:28:40.069201 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=9 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:28:48.040442 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=28 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:28:56.457020 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa45f0b-1767-48fa-91c7-6804024d6766 service swift] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:28:56.459709 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-3fa45f0b-1767-48fa-91c7-6804024d6766 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:28:56.580713 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa45f0b-1767-48fa-91c7-6804024d6766 service swift] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:28:56.587025 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c83fd1-7cfa-460c-8077-018bd5570eef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:28:56.587745 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c83fd1-7cfa-460c-8077-018bd5570eef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:28:56.588062 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c83fd1-7cfa-460c-8077-018bd5570eef admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:28:56.588970 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c83fd1-7cfa-460c-8077-018bd5570eef admin admin] PATH_INFO: `/v3/users/e2d913731d3d4c7f91741500dbe5892a` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:28:56.590005 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c83fd1-7cfa-460c-8077-018bd5570eef admin admin] RBAC: Authorizing `identity:delete_user(user_id=e2d913731d3d4c7f91741500dbe5892a)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:28:56.592293 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c83fd1-7cfa-460c-8077-018bd5570eef admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:28:56.837843 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-e5c83fd1-7cfa-460c-8077-018bd5570eef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e2d913731d3d4c7f91741500dbe5892a'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:28:56.935736 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-e5c83fd1-7cfa-460c-8077-018bd5570eef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e2d913731d3d4c7f91741500dbe5892a'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:28:56.936464 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-e5c83fd1-7cfa-460c-8077-018bd5570eef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e2d913731d3d4c7f91741500dbe5892a'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:28:56.949661 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-e5c83fd1-7cfa-460c-8077-018bd5570eef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e2d913731d3d4c7f91741500dbe5892a'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:28:56.950421 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-e5c83fd1-7cfa-460c-8077-018bd5570eef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e2d913731d3d4c7f91741500dbe5892a'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:28:56.964467 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1205/2416] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jun 29 11:28:56 2022] DELETE /identity/v3/users/e2d913731d3d4c7f91741500dbe5892a => generated 0 bytes in 509 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:28:56.969390 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da6a1ec-0520-4814-92d8-4dd3b17f7485 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:28:56.970150 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da6a1ec-0520-4814-92d8-4dd3b17f7485 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:28:56.970326 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da6a1ec-0520-4814-92d8-4dd3b17f7485 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:28:56.989771 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-8da6a1ec-0520-4814-92d8-4dd3b17f7485 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:28:56.991161 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-8da6a1ec-0520-4814-92d8-4dd3b17f7485 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:28:57.052514 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1212/2417] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:28:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:28:57.062162 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c83fd1-7cfa-460c-8077-018bd5570eef admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:28:57.063607 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-e5c83fd1-7cfa-460c-8077-018bd5570eef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:28:57.133366 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c83fd1-7cfa-460c-8077-018bd5570eef admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:28:57.151122 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80b4891-e2de-48c3-aae2-d7d1fd650957 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f4bb9f989e4024a5da28dced646b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:28:57.157401 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80b4891-e2de-48c3-aae2-d7d1fd650957 service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:28:57.157401 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80b4891-e2de-48c3-aae2-d7d1fd650957 service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:28:57.157401 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80b4891-e2de-48c3-aae2-d7d1fd650957 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:28:57.165532 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80b4891-e2de-48c3-aae2-d7d1fd650957 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:28:57.167300 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80b4891-e2de-48c3-aae2-d7d1fd650957 service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:28:57.183028 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1206/2418] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:28:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:28:57.616531 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da6a1ec-0520-4814-92d8-4dd3b17f7485 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:28:57.618465 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-8da6a1ec-0520-4814-92d8-4dd3b17f7485 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:28:57.632609 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da6a1ec-0520-4814-92d8-4dd3b17f7485 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:28:57.647090 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a502caa-db69-4539-8cf9-38f87f6ec84b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:28:57.647090 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a502caa-db69-4539-8cf9-38f87f6ec84b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:28:57.647090 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a502caa-db69-4539-8cf9-38f87f6ec84b admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:28:57.647090 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a502caa-db69-4539-8cf9-38f87f6ec84b admin admin] PATH_INFO: `/v3/projects/3eb664fdc4264ff783c647402adb0633` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:28:57.648343 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a502caa-db69-4539-8cf9-38f87f6ec84b admin admin] RBAC: Authorizing `identity:delete_project(project_id=3eb664fdc4264ff783c647402adb0633)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:28:57.648343 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a502caa-db69-4539-8cf9-38f87f6ec84b admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:28:57.788902 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-9a502caa-db69-4539-8cf9-38f87f6ec84b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3eb664fdc4264ff783c647402adb0633', 'request_id': 'req-9a502caa-db69-4539-8cf9-38f87f6ec84b'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:28:57.795178 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-9a502caa-db69-4539-8cf9-38f87f6ec84b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3eb664fdc4264ff783c647402adb0633', 'request_id': 'req-9a502caa-db69-4539-8cf9-38f87f6ec84b'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:28:57.802348 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-9a502caa-db69-4539-8cf9-38f87f6ec84b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3eb664fdc4264ff783c647402adb0633', 'request_id': 'req-9a502caa-db69-4539-8cf9-38f87f6ec84b'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:28:57.917817 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-9a502caa-db69-4539-8cf9-38f87f6ec84b admin admin] The token cache is being invalidate because project 3eb664fdc4264ff783c647402adb0633 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 11:28:57.925511 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-9a502caa-db69-4539-8cf9-38f87f6ec84b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:28:57.925511 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-9a502caa-db69-4539-8cf9-38f87f6ec84b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:28:57.925511 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1213/2419] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jun 29 11:28:57 2022] DELETE /identity/v3/projects/3eb664fdc4264ff783c647402adb0633 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:28:57.929655 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babc2c83-ce1d-455c-ab40-88b5c0c9a5ef None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:28:57.930061 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babc2c83-ce1d-455c-ab40-88b5c0c9a5ef None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:28:57.930213 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babc2c83-ce1d-455c-ab40-88b5c0c9a5ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:28:57.952636 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-babc2c83-ce1d-455c-ab40-88b5c0c9a5ef None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:28:57.952636 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-babc2c83-ce1d-455c-ab40-88b5c0c9a5ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:28:57.966841 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:28:58.012862 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1207/2420] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:28:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:28:58.020697 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e062eb4d-0730-4333-92e5-4cb1f3f4898a None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:28:58.021632 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e062eb4d-0730-4333-92e5-4cb1f3f4898a None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:28:58.021920 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e062eb4d-0730-4333-92e5-4cb1f3f4898a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:28:58.041272 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-e062eb4d-0730-4333-92e5-4cb1f3f4898a None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:28:58.042467 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-e062eb4d-0730-4333-92e5-4cb1f3f4898a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:28:58.051353 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1214/2421] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:28:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:28:58.055547 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babc2c83-ce1d-455c-ab40-88b5c0c9a5ef None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:28:58.069216 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babc2c83-ce1d-455c-ab40-88b5c0c9a5ef None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:28:58.070539 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f30bfb-d636-4605-ac58-6b2c8ddc73c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:28:58.071195 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f30bfb-d636-4605-ac58-6b2c8ddc73c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:28:58.071482 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f30bfb-d636-4605-ac58-6b2c8ddc73c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:28:58.071763 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f30bfb-d636-4605-ac58-6b2c8ddc73c3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:28:58.072529 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f30bfb-d636-4605-ac58-6b2c8ddc73c3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:28:58.072839 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f30bfb-d636-4605-ac58-6b2c8ddc73c3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:28:58.074434 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f30bfb-d636-4605-ac58-6b2c8ddc73c3 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:28:58.086750 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1208/2422] 10.222.0.44 () {58 vars in 1152 bytes} [Wed Jun 29 11:28:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:28:58.095221 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e062eb4d-0730-4333-92e5-4cb1f3f4898a None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:28:58.105369 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e062eb4d-0730-4333-92e5-4cb1f3f4898a None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:28:58.110442 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3b1a88-1600-4c65-99b9-270ac01f151e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:28:58.111089 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3b1a88-1600-4c65-99b9-270ac01f151e admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:28:58.111373 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3b1a88-1600-4c65-99b9-270ac01f151e admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:28:58.111651 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3b1a88-1600-4c65-99b9-270ac01f151e admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:28:58.112376 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3b1a88-1600-4c65-99b9-270ac01f151e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:28:58.114908 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3b1a88-1600-4c65-99b9-270ac01f151e admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:28:58.193284 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1215/2423] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jun 29 11:28:58 2022] POST /identity/v3/projects => generated 399 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:28:58.197487 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f30bfb-d636-4605-ac58-6b2c8ddc73c3 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:28:58.207906 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f30bfb-d636-4605-ac58-6b2c8ddc73c3 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:28:58.265244 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1b8323-a4ab-4041-8583-cf78b1c4c2ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:28:58.265925 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1b8323-a4ab-4041-8583-cf78b1c4c2ed admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:28:58.266195 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1b8323-a4ab-4041-8583-cf78b1c4c2ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:28:58.266471 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1b8323-a4ab-4041-8583-cf78b1c4c2ed admin admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:28:58.267166 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1b8323-a4ab-4041-8583-cf78b1c4c2ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:28:58.269118 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1b8323-a4ab-4041-8583-cf78b1c4c2ed admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:28:58.396516 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1209/2424] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jun 29 11:28:58 2022] POST /identity/v3/users => generated 300 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:28:58.397775 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3b1a88-1600-4c65-99b9-270ac01f151e admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:28:58.408305 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3b1a88-1600-4c65-99b9-270ac01f151e admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:28:58.413726 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8d67a6-2549-4d2a-bdd9-f736319d1e71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:28:58.414436 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8d67a6-2549-4d2a-bdd9-f736319d1e71 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:28:58.414729 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8d67a6-2549-4d2a-bdd9-f736319d1e71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:28:58.414976 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8d67a6-2549-4d2a-bdd9-f736319d1e71 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:28:58.415699 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8d67a6-2549-4d2a-bdd9-f736319d1e71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:28:58.417599 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8d67a6-2549-4d2a-bdd9-f736319d1e71 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:28:58.428551 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1216/2425] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:28:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 11:28:58.432750 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1b8323-a4ab-4041-8583-cf78b1c4c2ed admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:28:58.442849 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1b8323-a4ab-4041-8583-cf78b1c4c2ed admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:28:58.447466 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afcad09-ea2a-44ef-b041-4d079e79f0c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:28:58.448363 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afcad09-ea2a-44ef-b041-4d079e79f0c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:28:58.448623 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afcad09-ea2a-44ef-b041-4d079e79f0c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:28:58.448896 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afcad09-ea2a-44ef-b041-4d079e79f0c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:28:58.449752 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afcad09-ea2a-44ef-b041-4d079e79f0c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:28:58.451475 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afcad09-ea2a-44ef-b041-4d079e79f0c5 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:28:58.468014 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1210/2426] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:28:58 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 11:28:58.470169 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8d67a6-2549-4d2a-bdd9-f736319d1e71 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:28:58.481921 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8d67a6-2549-4d2a-bdd9-f736319d1e71 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:28:58.488014 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d0025d-6b74-44d9-b36d-349117ee6da4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:28:58.488906 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d0025d-6b74-44d9-b36d-349117ee6da4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:28:58.489212 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d0025d-6b74-44d9-b36d-349117ee6da4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:28:58.489530 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d0025d-6b74-44d9-b36d-349117ee6da4 admin admin] PATH_INFO: `/v3/projects/b2cffc9006044397acd959c4bffa46db/users/0bef7d7208174ea8902f66b11fae7ce1/roles/e1c88eebe911432cab6a07fbeb84653b` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:28:58.505888 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d0025d-6b74-44d9-b36d-349117ee6da4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2cffc9006044397acd959c4bffa46db, user_id=0bef7d7208174ea8902f66b11fae7ce1, role_id=e1c88eebe911432cab6a07fbeb84653b)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:28:58.508794 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d0025d-6b74-44d9-b36d-349117ee6da4 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:28:58.615265 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1217/2427] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jun 29 11:28:58 2022] PUT /identity/v3/projects/b2cffc9006044397acd959c4bffa46db/users/0bef7d7208174ea8902f66b11fae7ce1/roles/e1c88eebe911432cab6a07fbeb84653b => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 11:28:58.626489 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec4faa3-6c6c-4519-a40d-da642371b88b None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:28:58.626784 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec4faa3-6c6c-4519-a40d-da642371b88b None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:28:58.627052 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec4faa3-6c6c-4519-a40d-da642371b88b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:28:58.655074 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-0ec4faa3-6c6c-4519-a40d-da642371b88b None None] MFA Rules not processed for user `0bef7d7208174ea8902f66b11fae7ce1`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:28:58.656309 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-0ec4faa3-6c6c-4519-a40d-da642371b88b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:28:58.721351 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1211/2428] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:28:58 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 11:28:58.733376 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d0025d-6b74-44d9-b36d-349117ee6da4 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:28:58.735242 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-f9d0025d-6b74-44d9-b36d-349117ee6da4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:28:58.764259 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:28:58.819323 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d0025d-6b74-44d9-b36d-349117ee6da4 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:28:58.823151 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b416fcc3-3648-4bd1-80e6-97d8071a3966 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff7931b75a924394b1af733febce0834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:28:58.823794 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b416fcc3-3648-4bd1-80e6-97d8071a3966 service cinder] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:28:58.824109 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b416fcc3-3648-4bd1-80e6-97d8071a3966 service cinder] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:28:58.824386 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b416fcc3-3648-4bd1-80e6-97d8071a3966 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:28:58.833470 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b416fcc3-3648-4bd1-80e6-97d8071a3966 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:28:58.834885 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b416fcc3-3648-4bd1-80e6-97d8071a3966 service cinder] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:28:58.843613 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1218/2429] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:28:58 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:29:08.718485 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=89,keystone:INSERT=3 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:29:08.987297 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=26 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:29:09.950066 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec4faa3-6c6c-4519-a40d-da642371b88b None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:29:10.013201 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec4faa3-6c6c-4519-a40d-da642371b88b None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:29:10.018343 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce47e7f-29b7-401f-bbc8-d0d9d8bc4b96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:29:10.019314 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce47e7f-29b7-401f-bbc8-d0d9d8bc4b96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:29:10.019669 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce47e7f-29b7-401f-bbc8-d0d9d8bc4b96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:29:10.020026 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce47e7f-29b7-401f-bbc8-d0d9d8bc4b96 admin admin] PATH_INFO: `/v3/users/0bef7d7208174ea8902f66b11fae7ce1` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:29:10.021459 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce47e7f-29b7-401f-bbc8-d0d9d8bc4b96 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0bef7d7208174ea8902f66b11fae7ce1)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:29:10.023615 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce47e7f-29b7-401f-bbc8-d0d9d8bc4b96 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:29:10.217426 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-3ce47e7f-29b7-401f-bbc8-d0d9d8bc4b96 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0bef7d7208174ea8902f66b11fae7ce1'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:29:10.282381 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-3ce47e7f-29b7-401f-bbc8-d0d9d8bc4b96 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0bef7d7208174ea8902f66b11fae7ce1'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:29:10.283026 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-3ce47e7f-29b7-401f-bbc8-d0d9d8bc4b96 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0bef7d7208174ea8902f66b11fae7ce1'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:29:10.295786 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-3ce47e7f-29b7-401f-bbc8-d0d9d8bc4b96 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0bef7d7208174ea8902f66b11fae7ce1'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:29:10.296450 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-3ce47e7f-29b7-401f-bbc8-d0d9d8bc4b96 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0bef7d7208174ea8902f66b11fae7ce1'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:29:10.308454 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1212/2430] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jun 29 11:29:09 2022] DELETE /identity/v3/users/0bef7d7208174ea8902f66b11fae7ce1 => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:29:10.313064 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3147e023-655f-44a7-bea9-8b5b08b5c650 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:29:10.313513 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3147e023-655f-44a7-bea9-8b5b08b5c650 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:29:10.313626 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3147e023-655f-44a7-bea9-8b5b08b5c650 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:29:10.330840 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-3147e023-655f-44a7-bea9-8b5b08b5c650 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:29:10.332091 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-3147e023-655f-44a7-bea9-8b5b08b5c650 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:29:10.395044 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1219/2431] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:29:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:29:10.417679 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce47e7f-29b7-401f-bbc8-d0d9d8bc4b96 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:29:10.419569 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-3ce47e7f-29b7-401f-bbc8-d0d9d8bc4b96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:29:10.482920 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce47e7f-29b7-401f-bbc8-d0d9d8bc4b96 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:29:10.486700 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c664ba-3a1f-4eff-aa7a-fcb757c80814 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f4bb9f989e4024a5da28dced646b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:29:10.487352 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c664ba-3a1f-4eff-aa7a-fcb757c80814 service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:29:10.487639 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c664ba-3a1f-4eff-aa7a-fcb757c80814 service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:29:10.487955 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c664ba-3a1f-4eff-aa7a-fcb757c80814 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:29:10.497311 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c664ba-3a1f-4eff-aa7a-fcb757c80814 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:29:10.498776 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c664ba-3a1f-4eff-aa7a-fcb757c80814 service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:29:10.590490 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1213/2432] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:29:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:29:11.213743 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3147e023-655f-44a7-bea9-8b5b08b5c650 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:29:11.219889 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-3147e023-655f-44a7-bea9-8b5b08b5c650 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:29:11.236115 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3147e023-655f-44a7-bea9-8b5b08b5c650 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:29:11.240875 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab60e77e-7e6e-467f-b5be-f2da7e66be4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:29:11.242396 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab60e77e-7e6e-467f-b5be-f2da7e66be4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:29:11.242741 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab60e77e-7e6e-467f-b5be-f2da7e66be4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:29:11.243022 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab60e77e-7e6e-467f-b5be-f2da7e66be4f admin admin] PATH_INFO: `/v3/projects/b2cffc9006044397acd959c4bffa46db` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:29:11.244273 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab60e77e-7e6e-467f-b5be-f2da7e66be4f admin admin] RBAC: Authorizing `identity:delete_project(project_id=b2cffc9006044397acd959c4bffa46db)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:29:11.246194 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab60e77e-7e6e-467f-b5be-f2da7e66be4f admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:29:11.385970 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-ab60e77e-7e6e-467f-b5be-f2da7e66be4f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b2cffc9006044397acd959c4bffa46db', 'request_id': 'req-ab60e77e-7e6e-467f-b5be-f2da7e66be4f'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:29:11.392222 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-ab60e77e-7e6e-467f-b5be-f2da7e66be4f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b2cffc9006044397acd959c4bffa46db', 'request_id': 'req-ab60e77e-7e6e-467f-b5be-f2da7e66be4f'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:29:11.397940 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-ab60e77e-7e6e-467f-b5be-f2da7e66be4f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b2cffc9006044397acd959c4bffa46db', 'request_id': 'req-ab60e77e-7e6e-467f-b5be-f2da7e66be4f'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:29:11.501083 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-ab60e77e-7e6e-467f-b5be-f2da7e66be4f admin admin] The token cache is being invalidate because project b2cffc9006044397acd959c4bffa46db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 11:29:11.501380 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-ab60e77e-7e6e-467f-b5be-f2da7e66be4f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:29:11.502001 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-ab60e77e-7e6e-467f-b5be-f2da7e66be4f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:29:11.503814 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1220/2433] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jun 29 11:29:11 2022] DELETE /identity/v3/projects/b2cffc9006044397acd959c4bffa46db => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:29:15.421918 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c664ba-3a1f-4eff-aa7a-fcb757c80814 service neutron] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:29:15.431280 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c664ba-3a1f-4eff-aa7a-fcb757c80814 service neutron] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:29:15.448641 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:29:15.479690 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eae5e21-e3ce-44e7-b964-c0e226dd56f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f4bb9f989e4024a5da28dced646b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:29:15.480360 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eae5e21-e3ce-44e7-b964-c0e226dd56f1 service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:29:15.480645 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eae5e21-e3ce-44e7-b964-c0e226dd56f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:29:15.480920 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eae5e21-e3ce-44e7-b964-c0e226dd56f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:29:15.482704 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-2eae5e21-e3ce-44e7-b964-c0e226dd56f1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:29:15.551055 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eae5e21-e3ce-44e7-b964-c0e226dd56f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:29:15.552546 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eae5e21-e3ce-44e7-b964-c0e226dd56f1 service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:29:15.562435 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1214/2434] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:29:15 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:29:18.075985 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab60e77e-7e6e-467f-b5be-f2da7e66be4f admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:29:18.077804 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-ab60e77e-7e6e-467f-b5be-f2da7e66be4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:29:18.090864 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab60e77e-7e6e-467f-b5be-f2da7e66be4f admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:29:18.094618 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bf4eef-d88e-46bb-8798-cd77a2ea2b50 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f4bb9f989e4024a5da28dced646b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:29:18.095271 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bf4eef-d88e-46bb-8798-cd77a2ea2b50 service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:29:18.095559 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bf4eef-d88e-46bb-8798-cd77a2ea2b50 service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:29:18.095865 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bf4eef-d88e-46bb-8798-cd77a2ea2b50 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:29:18.097956 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-10bf4eef-d88e-46bb-8798-cd77a2ea2b50 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:29:18.170678 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bf4eef-d88e-46bb-8798-cd77a2ea2b50 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:29:18.172429 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bf4eef-d88e-46bb-8798-cd77a2ea2b50 service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:29:18.184871 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1221/2435] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:29:18 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:29:25.569864 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=40 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:29:28.178535 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:29:55.025569 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eae5e21-e3ce-44e7-b964-c0e226dd56f1 service neutron] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:29:55.028008 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-2eae5e21-e3ce-44e7-b964-c0e226dd56f1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:29:55.101544 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eae5e21-e3ce-44e7-b964-c0e226dd56f1 service neutron] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:29:55.107045 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2830aae0-615b-4ad3-abf6-bfa707063655 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '796aeeff16df4739bddec1f6dee61bfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:29:55.107717 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2830aae0-615b-4ad3-abf6-bfa707063655 service nova] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:29:55.108545 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2830aae0-615b-4ad3-abf6-bfa707063655 service nova] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:29:55.108821 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2830aae0-615b-4ad3-abf6-bfa707063655 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:29:55.110558 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-2830aae0-615b-4ad3-abf6-bfa707063655 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:29:55.166242 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2830aae0-615b-4ad3-abf6-bfa707063655 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:29:55.168220 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2830aae0-615b-4ad3-abf6-bfa707063655 service nova] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:29:55.177678 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1215/2436] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:29:55 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:29:55.226562 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bf4eef-d88e-46bb-8798-cd77a2ea2b50 service neutron] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:29:55.227986 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-10bf4eef-d88e-46bb-8798-cd77a2ea2b50 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:29:55.283773 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bf4eef-d88e-46bb-8798-cd77a2ea2b50 service neutron] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:29:55.287299 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e7b0ea-6551-4640-b367-e68e96b34f61 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff7931b75a924394b1af733febce0834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:29:55.287974 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e7b0ea-6551-4640-b367-e68e96b34f61 service cinder] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:29:55.288262 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e7b0ea-6551-4640-b367-e68e96b34f61 service cinder] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:29:55.288547 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e7b0ea-6551-4640-b367-e68e96b34f61 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:29:55.290287 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-c8e7b0ea-6551-4640-b367-e68e96b34f61 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:29:55.305110 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e7b0ea-6551-4640-b367-e68e96b34f61 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:29:55.306649 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e7b0ea-6551-4640-b367-e68e96b34f61 service cinder] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:29:55.318439 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1222/2437] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:29:55 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:30:05.091457 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2830aae0-615b-4ad3-abf6-bfa707063655 service nova] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:30:05.094956 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-2830aae0-615b-4ad3-abf6-bfa707063655 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:30:05.149916 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2830aae0-615b-4ad3-abf6-bfa707063655 service nova] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:30:05.153478 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157256eb-86d2-4454-a5dc-5bfa5060802c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede3a17c7063432e8ee66462766584f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:30:05.154682 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157256eb-86d2-4454-a5dc-5bfa5060802c service glance] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:30:05.154682 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157256eb-86d2-4454-a5dc-5bfa5060802c service glance] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:30:05.154682 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157256eb-86d2-4454-a5dc-5bfa5060802c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:30:05.156494 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-157256eb-86d2-4454-a5dc-5bfa5060802c service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:30:05.170135 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157256eb-86d2-4454-a5dc-5bfa5060802c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:30:05.171567 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157256eb-86d2-4454-a5dc-5bfa5060802c service glance] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:30:05.183312 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1216/2438] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:30:05 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:30:05.313551 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=26 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:30:15.178230 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=82 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:30:18.832791 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e7b0ea-6551-4640-b367-e68e96b34f61 service cinder] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:30:18.834782 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-c8e7b0ea-6551-4640-b367-e68e96b34f61 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:30:18.847591 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e7b0ea-6551-4640-b367-e68e96b34f61 service cinder] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:30:18.855853 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f876d45c-f051-4436-876c-cfeec72bb902 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f4bb9f989e4024a5da28dced646b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:30:18.855853 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f876d45c-f051-4436-876c-cfeec72bb902 service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:30:18.855853 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f876d45c-f051-4436-876c-cfeec72bb902 service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:30:18.855853 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f876d45c-f051-4436-876c-cfeec72bb902 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:30:18.856392 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-f876d45c-f051-4436-876c-cfeec72bb902 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:30:18.866612 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f876d45c-f051-4436-876c-cfeec72bb902 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:30:18.870275 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f876d45c-f051-4436-876c-cfeec72bb902 service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:30:18.884999 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1223/2439] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:30:18 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:30:20.992479 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157256eb-86d2-4454-a5dc-5bfa5060802c service glance] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:30:20.994018 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-157256eb-86d2-4454-a5dc-5bfa5060802c service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:30:21.010652 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157256eb-86d2-4454-a5dc-5bfa5060802c service glance] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:30:21.016679 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a37b096-e51a-4266-8e8f-6710f545a1a4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '796aeeff16df4739bddec1f6dee61bfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:30:21.017838 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a37b096-e51a-4266-8e8f-6710f545a1a4 service nova] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:30:21.018116 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a37b096-e51a-4266-8e8f-6710f545a1a4 service nova] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:30:21.018386 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a37b096-e51a-4266-8e8f-6710f545a1a4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:30:21.020204 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-8a37b096-e51a-4266-8e8f-6710f545a1a4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:30:21.034018 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a37b096-e51a-4266-8e8f-6710f545a1a4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:30:21.038023 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a37b096-e51a-4266-8e8f-6710f545a1a4 service nova] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:30:21.046866 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1217/2440] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:30:20 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:30:22.594841 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353c9e3b-54d8-4a54-9385-60ec7d871a5c None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:30:22.595344 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353c9e3b-54d8-4a54-9385-60ec7d871a5c None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:30:22.595538 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353c9e3b-54d8-4a54-9385-60ec7d871a5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:30:22.612666 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-353c9e3b-54d8-4a54-9385-60ec7d871a5c None None] MFA Rules not processed for user `8effcd6f6be64f14b3c8bc71f8866a99`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:30:22.613853 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-353c9e3b-54d8-4a54-9385-60ec7d871a5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:30:22.620649 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1224/2441] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:30:22 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:30:27.559688 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb9dd25-ca90-4a38-8951-2e6c2ce85b41 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:30:27.560206 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb9dd25-ca90-4a38-8951-2e6c2ce85b41 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:30:27.560330 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb9dd25-ca90-4a38-8951-2e6c2ce85b41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:30:27.577568 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-feb9dd25-ca90-4a38-8951-2e6c2ce85b41 None None] MFA Rules not processed for user `55487c2107a44c15b656b4865ee0d4fa`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:30:27.578912 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-feb9dd25-ca90-4a38-8951-2e6c2ce85b41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:30:27.634909 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1218/2442] 10.222.0.44 () {58 vars in 1009 bytes} [Wed Jun 29 11:30:27 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:30:27.676605 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353c9e3b-54d8-4a54-9385-60ec7d871a5c None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:30:27.678020 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-353c9e3b-54d8-4a54-9385-60ec7d871a5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:30:27.747224 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353c9e3b-54d8-4a54-9385-60ec7d871a5c None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:30:27.752208 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335894cc-c092-4454-b3c8-ddd51be31cfb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3b38493ba904759badc800e396befda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:30:27.752858 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335894cc-c092-4454-b3c8-ddd51be31cfb service swift] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:30:27.753146 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335894cc-c092-4454-b3c8-ddd51be31cfb service swift] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:30:27.753418 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335894cc-c092-4454-b3c8-ddd51be31cfb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:30:27.762220 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335894cc-c092-4454-b3c8-ddd51be31cfb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:30:27.763809 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335894cc-c092-4454-b3c8-ddd51be31cfb service swift] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:30:27.773334 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1225/2443] 10.222.0.44 () {60 vars in 1384 bytes} [Wed Jun 29 11:30:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 29 11:30:29.914230 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb9dd25-ca90-4a38-8951-2e6c2ce85b41 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:30:29.933447 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb9dd25-ca90-4a38-8951-2e6c2ce85b41 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:30:29.960486 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fdac5e-1df5-48e4-bc8e-575cd02b4b61 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '796aeeff16df4739bddec1f6dee61bfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:30:29.961551 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fdac5e-1df5-48e4-bc8e-575cd02b4b61 service nova] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:30:29.962032 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fdac5e-1df5-48e4-bc8e-575cd02b4b61 service nova] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:30:29.962511 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fdac5e-1df5-48e4-bc8e-575cd02b4b61 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:30:29.964601 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-35fdac5e-1df5-48e4-bc8e-575cd02b4b61 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:30:29.980416 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fdac5e-1df5-48e4-bc8e-575cd02b4b61 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:30:29.982574 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fdac5e-1df5-48e4-bc8e-575cd02b4b61 service nova] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:30:30.001447 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1219/2444] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:30:29 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:30:37.769334 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=41 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:30:39.014241 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335894cc-c092-4454-b3c8-ddd51be31cfb service swift] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:30:39.026701 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335894cc-c092-4454-b3c8-ddd51be31cfb service swift] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:30:39.032887 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22733b5-adcb-4a06-b270-353e088fdf50 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '796aeeff16df4739bddec1f6dee61bfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:30:39.033545 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22733b5-adcb-4a06-b270-353e088fdf50 service nova] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:30:39.033831 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22733b5-adcb-4a06-b270-353e088fdf50 service nova] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:30:39.034109 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22733b5-adcb-4a06-b270-353e088fdf50 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:30:39.036051 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-b22733b5-adcb-4a06-b270-353e088fdf50 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:30:39.105998 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22733b5-adcb-4a06-b270-353e088fdf50 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:30:39.107652 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22733b5-adcb-4a06-b270-353e088fdf50 service nova] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:30:39.119279 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1226/2445] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:30:39 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:30:39.990386 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=37 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:30:46.146013 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fdac5e-1df5-48e4-bc8e-575cd02b4b61 service nova] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:30:46.155666 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fdac5e-1df5-48e4-bc8e-575cd02b4b61 service nova] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:30:46.159917 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d22ccff-c4c3-472d-9b3c-a8ea49eccb1c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f4bb9f989e4024a5da28dced646b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:30:46.160547 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d22ccff-c4c3-472d-9b3c-a8ea49eccb1c service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:30:46.160833 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d22ccff-c4c3-472d-9b3c-a8ea49eccb1c service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:30:46.161111 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d22ccff-c4c3-472d-9b3c-a8ea49eccb1c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:30:46.163006 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-5d22ccff-c4c3-472d-9b3c-a8ea49eccb1c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:30:46.172897 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d22ccff-c4c3-472d-9b3c-a8ea49eccb1c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:30:46.174933 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d22ccff-c4c3-472d-9b3c-a8ea49eccb1c service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:30:46.182810 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1220/2446] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:30:46 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:30:46.388760 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a6ebbc-f136-4cc4-ab06-72df59cbbfcf None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:30:46.389336 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a6ebbc-f136-4cc4-ab06-72df59cbbfcf None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:30:46.389336 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a6ebbc-f136-4cc4-ab06-72df59cbbfcf None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:30:46.390720 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1227/2447] 10.222.0.44 () {54 vars in 921 bytes} [Wed Jun 29 11:30:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:30:46.397585 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92087ea2-df9b-42cf-a7c2-05962de0ffd9 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:30:46.397907 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92087ea2-df9b-42cf-a7c2-05962de0ffd9 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:30:46.398187 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92087ea2-df9b-42cf-a7c2-05962de0ffd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:30:46.436177 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-92087ea2-df9b-42cf-a7c2-05962de0ffd9 None None] MFA Rules not processed for user `796aeeff16df4739bddec1f6dee61bfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:30:46.437433 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-92087ea2-df9b-42cf-a7c2-05962de0ffd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:30:46.446896 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1221/2448] 10.222.0.44 () {58 vars in 1018 bytes} [Wed Jun 29 11:30:46 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:30:46.458765 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a6ebbc-f136-4cc4-ab06-72df59cbbfcf None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:30:46.470028 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a6ebbc-f136-4cc4-ab06-72df59cbbfcf None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:30:46.474293 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6973f31b-9e1d-4676-a1cc-59e2e418a81f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede3a17c7063432e8ee66462766584f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:30:46.475197 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6973f31b-9e1d-4676-a1cc-59e2e418a81f service glance] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:30:46.475481 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6973f31b-9e1d-4676-a1cc-59e2e418a81f service glance] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:30:46.475761 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6973f31b-9e1d-4676-a1cc-59e2e418a81f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:30:46.486969 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6973f31b-9e1d-4676-a1cc-59e2e418a81f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:30:46.488440 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6973f31b-9e1d-4676-a1cc-59e2e418a81f service glance] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:30:46.499260 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1228/2449] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:30:46 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 11:30:46.685414 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eaf924-505e-4bb7-9796-3ad047602718 None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:30:46.687265 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eaf924-505e-4bb7-9796-3ad047602718 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:30:46.687635 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eaf924-505e-4bb7-9796-3ad047602718 None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:30:46.689852 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1222/2450] 10.222.0.44 () {54 vars in 921 bytes} [Wed Jun 29 11:30:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 29 11:30:46.694786 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b58034-35e2-41ca-8d19-b28fd8b5e20f None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:30:46.695163 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b58034-35e2-41ca-8d19-b28fd8b5e20f None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:30:46.695458 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b58034-35e2-41ca-8d19-b28fd8b5e20f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:30:46.715112 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-99b58034-35e2-41ca-8d19-b28fd8b5e20f None None] MFA Rules not processed for user `b5f4bb9f989e4024a5da28dced646b92`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:30:46.716505 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-99b58034-35e2-41ca-8d19-b28fd8b5e20f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:30:46.725747 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1229/2451] 10.222.0.44 () {58 vars in 1018 bytes} [Wed Jun 29 11:30:46 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:30:46.738361 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99eaf924-505e-4bb7-9796-3ad047602718 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:30:46.749648 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99eaf924-505e-4bb7-9796-3ad047602718 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:30:46.753846 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe1cadc-9702-40ec-86c6-2753e98708f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f4bb9f989e4024a5da28dced646b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:30:46.754487 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe1cadc-9702-40ec-86c6-2753e98708f4 service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:30:46.754776 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe1cadc-9702-40ec-86c6-2753e98708f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:30:46.755056 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe1cadc-9702-40ec-86c6-2753e98708f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:30:46.762421 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe1cadc-9702-40ec-86c6-2753e98708f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:30:46.763854 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe1cadc-9702-40ec-86c6-2753e98708f4 service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:30:46.773585 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1223/2452] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:30:46 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:30:46.857293 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6426683d-94b0-4633-a559-099e30abc7aa None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:30:46.857653 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6426683d-94b0-4633-a559-099e30abc7aa None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:30:46.858002 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6426683d-94b0-4633-a559-099e30abc7aa None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:30:46.861162 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1230/2453] 10.222.0.44 () {54 vars in 921 bytes} [Wed Jun 29 11:30:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 11:30:46.871092 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7148de-36d5-4e48-b864-1b7ffa26eaa1 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:30:46.871311 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7148de-36d5-4e48-b864-1b7ffa26eaa1 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:30:46.871591 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7148de-36d5-4e48-b864-1b7ffa26eaa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:30:46.890293 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-ba7148de-36d5-4e48-b864-1b7ffa26eaa1 None None] MFA Rules not processed for user `796aeeff16df4739bddec1f6dee61bfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:30:46.891543 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-ba7148de-36d5-4e48-b864-1b7ffa26eaa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:30:46.901532 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1224/2454] 10.222.0.44 () {58 vars in 1018 bytes} [Wed Jun 29 11:30:46 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:30:46.921119 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6426683d-94b0-4633-a559-099e30abc7aa None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:30:46.932899 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6426683d-94b0-4633-a559-099e30abc7aa None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:30:46.937639 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52231e9-2bf1-4e94-a4e7-076acb5c5cb7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f4bb9f989e4024a5da28dced646b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:30:46.938674 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52231e9-2bf1-4e94-a4e7-076acb5c5cb7 service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:30:46.939759 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52231e9-2bf1-4e94-a4e7-076acb5c5cb7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:30:46.940071 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52231e9-2bf1-4e94-a4e7-076acb5c5cb7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:30:46.957535 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52231e9-2bf1-4e94-a4e7-076acb5c5cb7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:30:46.958972 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52231e9-2bf1-4e94-a4e7-076acb5c5cb7 service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:30:46.971737 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1231/2455] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:30:46 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:30:47.040218 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7148de-36d5-4e48-b864-1b7ffa26eaa1 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:30:47.042140 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-ba7148de-36d5-4e48-b864-1b7ffa26eaa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:30:47.112782 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7148de-36d5-4e48-b864-1b7ffa26eaa1 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:30:47.117712 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b48ac2a-3fd0-40dc-ae9a-a1bc414aa5c4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a0505342b704d02977a5125957c2c9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:30:47.118356 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b48ac2a-3fd0-40dc-ae9a-a1bc414aa5c4 service placement] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:30:47.118637 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b48ac2a-3fd0-40dc-ae9a-a1bc414aa5c4 service placement] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:30:47.118912 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b48ac2a-3fd0-40dc-ae9a-a1bc414aa5c4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:30:47.120741 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-3b48ac2a-3fd0-40dc-ae9a-a1bc414aa5c4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:30:47.132920 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b48ac2a-3fd0-40dc-ae9a-a1bc414aa5c4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:30:47.134494 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b48ac2a-3fd0-40dc-ae9a-a1bc414aa5c4 service placement] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:30:47.144733 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1225/2456] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:30:47 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:30:51.137164 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c957c8f-5410-41e2-adc6-523ffabd50be None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:30:51.137623 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c957c8f-5410-41e2-adc6-523ffabd50be None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:30:51.137848 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c957c8f-5410-41e2-adc6-523ffabd50be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:30:51.156198 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-6c957c8f-5410-41e2-adc6-523ffabd50be None None] MFA Rules not processed for user `55487c2107a44c15b656b4865ee0d4fa`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:30:51.157520 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-6c957c8f-5410-41e2-adc6-523ffabd50be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:30:51.164913 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1232/2457] 10.222.0.44 () {58 vars in 1009 bytes} [Wed Jun 29 11:30:51 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:30:51.181849 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b48ac2a-3fd0-40dc-ae9a-a1bc414aa5c4 service placement] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:30:51.193846 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b48ac2a-3fd0-40dc-ae9a-a1bc414aa5c4 service placement] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:30:51.198888 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9fb448-2ffd-4c27-b158-bfb6f2184ff7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3b38493ba904759badc800e396befda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:30:51.199527 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9fb448-2ffd-4c27-b158-bfb6f2184ff7 service swift] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:30:51.199804 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9fb448-2ffd-4c27-b158-bfb6f2184ff7 service swift] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:30:51.200111 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9fb448-2ffd-4c27-b158-bfb6f2184ff7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:30:51.208830 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9fb448-2ffd-4c27-b158-bfb6f2184ff7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:30:51.210394 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9fb448-2ffd-4c27-b158-bfb6f2184ff7 service swift] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:30:51.218237 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1226/2458] 10.222.0.44 () {60 vars in 1384 bytes} [Wed Jun 29 11:30:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 29 11:31:01.150943 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=46 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:31:01.215034 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=62 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:31:18.283331 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0beb576-2297-4312-97b5-582ee678e968 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:31:18.283331 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0beb576-2297-4312-97b5-582ee678e968 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:31:18.283331 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0beb576-2297-4312-97b5-582ee678e968 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:31:18.283331 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-f0beb576-2297-4312-97b5-582ee678e968 None None] MFA Rules not processed for user `55487c2107a44c15b656b4865ee0d4fa`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:31:18.283331 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-f0beb576-2297-4312-97b5-582ee678e968 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:31:18.284633 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1233/2459] 10.222.0.44 () {58 vars in 967 bytes} [Wed Jun 29 11:31:18 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:31:18.298984 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9fb448-2ffd-4c27-b158-bfb6f2184ff7 service swift] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:31:18.307790 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9fb448-2ffd-4c27-b158-bfb6f2184ff7 service swift] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:31:18.312953 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f90a00b-5853-4654-a84a-28e0278df946 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3b38493ba904759badc800e396befda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:31:18.313586 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f90a00b-5853-4654-a84a-28e0278df946 service swift] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:31:18.313872 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f90a00b-5853-4654-a84a-28e0278df946 service swift] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:31:18.314150 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f90a00b-5853-4654-a84a-28e0278df946 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:31:18.322623 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f90a00b-5853-4654-a84a-28e0278df946 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:31:18.324247 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f90a00b-5853-4654-a84a-28e0278df946 service swift] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:31:18.332945 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1227/2460] 10.222.0.44 () {60 vars in 1384 bytes} [Wed Jun 29 11:31:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 29 11:31:28.270309 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=3 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:31:28.329467 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=6 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:31:59.015005 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0beb576-2297-4312-97b5-582ee678e968 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:31:59.028905 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0beb576-2297-4312-97b5-582ee678e968 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:31:59.034436 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb25508-857b-45e8-aa9c-98b65f45c86d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '796aeeff16df4739bddec1f6dee61bfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:31:59.035103 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb25508-857b-45e8-aa9c-98b65f45c86d service nova] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:31:59.035404 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb25508-857b-45e8-aa9c-98b65f45c86d service nova] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:31:59.035662 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb25508-857b-45e8-aa9c-98b65f45c86d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:31:59.037708 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-efb25508-857b-45e8-aa9c-98b65f45c86d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:31:59.050866 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb25508-857b-45e8-aa9c-98b65f45c86d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:31:59.052748 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb25508-857b-45e8-aa9c-98b65f45c86d service nova] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:31:59.071002 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1234/2461] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:31:59 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 11:31:59.736820 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f90a00b-5853-4654-a84a-28e0278df946 service swift] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:31:59.747106 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f90a00b-5853-4654-a84a-28e0278df946 service swift] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:31:59.750799 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab098f9-7729-4dd3-acc8-fc5f2a5583f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f4bb9f989e4024a5da28dced646b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:31:59.751448 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab098f9-7729-4dd3-acc8-fc5f2a5583f7 service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:31:59.751726 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab098f9-7729-4dd3-acc8-fc5f2a5583f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:31:59.752042 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab098f9-7729-4dd3-acc8-fc5f2a5583f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:31:59.753840 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-0ab098f9-7729-4dd3-acc8-fc5f2a5583f7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:31:59.827846 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab098f9-7729-4dd3-acc8-fc5f2a5583f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:31:59.829572 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab098f9-7729-4dd3-acc8-fc5f2a5583f7 service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:31:59.840863 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1228/2462] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:31:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:32:00.679480 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991b47dc-db2b-48f9-b0c6-5ea0f3549e33 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:32:00.679957 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991b47dc-db2b-48f9-b0c6-5ea0f3549e33 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:32:00.680080 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991b47dc-db2b-48f9-b0c6-5ea0f3549e33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:32:00.697968 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-991b47dc-db2b-48f9-b0c6-5ea0f3549e33 None None] MFA Rules not processed for user `e91cc0b93f484f759233469ec5f37bb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:32:00.699197 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-991b47dc-db2b-48f9-b0c6-5ea0f3549e33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:32:00.758757 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1235/2463] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:32:00 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:32:03.539139 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab098f9-7729-4dd3-acc8-fc5f2a5583f7 service neutron] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:32:03.540756 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-0ab098f9-7729-4dd3-acc8-fc5f2a5583f7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:32:03.555089 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab098f9-7729-4dd3-acc8-fc5f2a5583f7 service neutron] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:32:03.560635 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5638ce82-6e52-434c-bae0-b13cd7cbfc96 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a0505342b704d02977a5125957c2c9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:32:03.561279 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5638ce82-6e52-434c-bae0-b13cd7cbfc96 service placement] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:32:03.561561 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5638ce82-6e52-434c-bae0-b13cd7cbfc96 service placement] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:32:03.561837 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5638ce82-6e52-434c-bae0-b13cd7cbfc96 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:32:03.571555 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-5638ce82-6e52-434c-bae0-b13cd7cbfc96 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:32:03.584769 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5638ce82-6e52-434c-bae0-b13cd7cbfc96 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:32:03.586490 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5638ce82-6e52-434c-bae0-b13cd7cbfc96 service placement] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:32:03.606926 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1229/2464] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:32:03 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:32:03.662102 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991b47dc-db2b-48f9-b0c6-5ea0f3549e33 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:32:03.663250 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-991b47dc-db2b-48f9-b0c6-5ea0f3549e33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:32:03.677870 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991b47dc-db2b-48f9-b0c6-5ea0f3549e33 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:32:03.685196 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f3d1ce-87d1-4904-84ae-cb230a457afe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:32:03.685196 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f3d1ce-87d1-4904-84ae-cb230a457afe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:32:03.685196 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f3d1ce-87d1-4904-84ae-cb230a457afe admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:32:03.685196 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f3d1ce-87d1-4904-84ae-cb230a457afe admin admin] PATH_INFO: `/v3/users/8effcd6f6be64f14b3c8bc71f8866a99` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:32:03.686168 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f3d1ce-87d1-4904-84ae-cb230a457afe admin admin] RBAC: Authorizing `identity:delete_user(user_id=8effcd6f6be64f14b3c8bc71f8866a99)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:32:03.688222 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f3d1ce-87d1-4904-84ae-cb230a457afe admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:32:03.863121 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-c6f3d1ce-87d1-4904-84ae-cb230a457afe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8effcd6f6be64f14b3c8bc71f8866a99'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:32:03.863712 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-c6f3d1ce-87d1-4904-84ae-cb230a457afe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8effcd6f6be64f14b3c8bc71f8866a99'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:32:03.917938 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-c6f3d1ce-87d1-4904-84ae-cb230a457afe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8effcd6f6be64f14b3c8bc71f8866a99'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:32:03.925545 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-c6f3d1ce-87d1-4904-84ae-cb230a457afe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8effcd6f6be64f14b3c8bc71f8866a99'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:32:03.926185 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-c6f3d1ce-87d1-4904-84ae-cb230a457afe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8effcd6f6be64f14b3c8bc71f8866a99'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:32:03.944723 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1236/2465] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jun 29 11:32:03 2022] DELETE /identity/v3/users/8effcd6f6be64f14b3c8bc71f8866a99 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:32:04.176651 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5638ce82-6e52-434c-bae0-b13cd7cbfc96 service placement] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:32:04.188865 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5638ce82-6e52-434c-bae0-b13cd7cbfc96 service placement] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:32:04.253354 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69280a39-c82b-4131-af5c-63d8cb0f245c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:32:04.254244 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69280a39-c82b-4131-af5c-63d8cb0f245c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:32:04.254526 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69280a39-c82b-4131-af5c-63d8cb0f245c admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:32:04.254802 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69280a39-c82b-4131-af5c-63d8cb0f245c admin admin] PATH_INFO: `/v3/projects/cbfba915f70342afb8267e34abc24bfe` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:32:04.256051 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69280a39-c82b-4131-af5c-63d8cb0f245c admin admin] RBAC: Authorizing `identity:delete_project(project_id=cbfba915f70342afb8267e34abc24bfe)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:32:04.257959 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69280a39-c82b-4131-af5c-63d8cb0f245c admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:32:04.371593 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-69280a39-c82b-4131-af5c-63d8cb0f245c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cbfba915f70342afb8267e34abc24bfe', 'request_id': 'req-69280a39-c82b-4131-af5c-63d8cb0f245c'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:32:04.447214 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-69280a39-c82b-4131-af5c-63d8cb0f245c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cbfba915f70342afb8267e34abc24bfe', 'request_id': 'req-69280a39-c82b-4131-af5c-63d8cb0f245c'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:32:04.456788 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-69280a39-c82b-4131-af5c-63d8cb0f245c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cbfba915f70342afb8267e34abc24bfe', 'request_id': 'req-69280a39-c82b-4131-af5c-63d8cb0f245c'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:32:04.471696 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-69280a39-c82b-4131-af5c-63d8cb0f245c admin admin] The token cache is being invalidate because project cbfba915f70342afb8267e34abc24bfe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 11:32:04.472035 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-69280a39-c82b-4131-af5c-63d8cb0f245c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:32:04.472699 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-69280a39-c82b-4131-af5c-63d8cb0f245c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:32:04.474319 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1230/2466] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jun 29 11:32:04 2022] DELETE /identity/v3/projects/cbfba915f70342afb8267e34abc24bfe => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:32:04.479584 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22d998f-073a-446c-b8bb-e09992ef01a5 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:32:04.479900 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22d998f-073a-446c-b8bb-e09992ef01a5 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:32:04.480182 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22d998f-073a-446c-b8bb-e09992ef01a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:32:04.499500 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-c22d998f-073a-446c-b8bb-e09992ef01a5 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:32:04.500724 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-c22d998f-073a-446c-b8bb-e09992ef01a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:32:04.564288 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1237/2467] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:32:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:32:04.576713 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be8ff38-dbf1-4a81-8729-bcbd727751d9 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:32:04.576713 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be8ff38-dbf1-4a81-8729-bcbd727751d9 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:32:04.576713 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be8ff38-dbf1-4a81-8729-bcbd727751d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:32:04.592505 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-1be8ff38-dbf1-4a81-8729-bcbd727751d9 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:32:04.594648 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-1be8ff38-dbf1-4a81-8729-bcbd727751d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:32:04.601576 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1231/2468] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:32:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:32:04.606097 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22d998f-073a-446c-b8bb-e09992ef01a5 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:32:04.612496 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:32:04.616687 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22d998f-073a-446c-b8bb-e09992ef01a5 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:32:04.621320 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bf8063-99a4-4051-961e-50f1de061541 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:32:04.623363 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bf8063-99a4-4051-961e-50f1de061541 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:32:04.623363 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bf8063-99a4-4051-961e-50f1de061541 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:32:04.623363 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bf8063-99a4-4051-961e-50f1de061541 admin admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:32:04.623363 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bf8063-99a4-4051-961e-50f1de061541 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:32:04.623732 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bf8063-99a4-4051-961e-50f1de061541 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:32:04.625179 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bf8063-99a4-4051-961e-50f1de061541 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:32:04.637618 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1238/2469] 10.222.0.44 () {58 vars in 1152 bytes} [Wed Jun 29 11:32:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:32:04.642015 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be8ff38-dbf1-4a81-8729-bcbd727751d9 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:32:04.647916 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:32:04.652595 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be8ff38-dbf1-4a81-8729-bcbd727751d9 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:32:04.657610 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd89f3f-5764-46bc-93c8-50287376c714 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:32:04.658335 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd89f3f-5764-46bc-93c8-50287376c714 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:32:04.658734 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd89f3f-5764-46bc-93c8-50287376c714 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:32:04.659018 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd89f3f-5764-46bc-93c8-50287376c714 admin admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:32:04.659717 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd89f3f-5764-46bc-93c8-50287376c714 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:32:04.661804 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd89f3f-5764-46bc-93c8-50287376c714 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:32:04.793917 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1232/2470] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jun 29 11:32:04 2022] POST /identity/v3/projects => generated 383 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:32:04.798140 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bf8063-99a4-4051-961e-50f1de061541 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:32:04.809128 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bf8063-99a4-4051-961e-50f1de061541 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:32:04.875622 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb88e10-370a-489a-9b86-9610f3bc9fc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:32:04.876334 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb88e10-370a-489a-9b86-9610f3bc9fc5 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:32:04.876675 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb88e10-370a-489a-9b86-9610f3bc9fc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:32:04.877214 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb88e10-370a-489a-9b86-9610f3bc9fc5 admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:32:04.878354 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb88e10-370a-489a-9b86-9610f3bc9fc5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:32:04.880884 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb88e10-370a-489a-9b86-9610f3bc9fc5 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:32:04.979418 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1239/2471] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jun 29 11:32:04 2022] POST /identity/v3/users => generated 292 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:32:04.980599 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd89f3f-5764-46bc-93c8-50287376c714 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:32:04.991868 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd89f3f-5764-46bc-93c8-50287376c714 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:32:04.998903 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c4bfbc-8795-4295-bac3-54a1744c543b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:32:04.999619 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c4bfbc-8795-4295-bac3-54a1744c543b admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:32:04.999940 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c4bfbc-8795-4295-bac3-54a1744c543b admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:32:05.000230 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c4bfbc-8795-4295-bac3-54a1744c543b admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:32:05.000969 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c4bfbc-8795-4295-bac3-54a1744c543b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:32:05.002774 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c4bfbc-8795-4295-bac3-54a1744c543b admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:32:05.014883 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1233/2472] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:32:04 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 11:32:05.019055 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb88e10-370a-489a-9b86-9610f3bc9fc5 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:32:05.032674 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb88e10-370a-489a-9b86-9610f3bc9fc5 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:32:05.037930 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8aa5a2a-bab9-4c2b-a4d3-3c466513c388 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:32:05.038770 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8aa5a2a-bab9-4c2b-a4d3-3c466513c388 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:32:05.039059 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8aa5a2a-bab9-4c2b-a4d3-3c466513c388 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:32:05.039337 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8aa5a2a-bab9-4c2b-a4d3-3c466513c388 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:32:05.040126 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8aa5a2a-bab9-4c2b-a4d3-3c466513c388 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:32:05.041762 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8aa5a2a-bab9-4c2b-a4d3-3c466513c388 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:32:05.052417 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1240/2473] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:32:05 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 11:32:05.056557 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c4bfbc-8795-4295-bac3-54a1744c543b admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:32:05.067786 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c4bfbc-8795-4295-bac3-54a1744c543b admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:32:05.074794 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ce5940-798c-4820-a4fa-21a3445baa26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:32:05.075692 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ce5940-798c-4820-a4fa-21a3445baa26 admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:32:05.076052 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ce5940-798c-4820-a4fa-21a3445baa26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:32:05.076328 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ce5940-798c-4820-a4fa-21a3445baa26 admin admin] PATH_INFO: `/v3/projects/7e39c9a1ff3c46dcbe4464f57534e5ea/users/cf7bc4681c954c158a7051046bd9aa47/roles/e1c88eebe911432cab6a07fbeb84653b` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:32:05.092470 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ce5940-798c-4820-a4fa-21a3445baa26 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e39c9a1ff3c46dcbe4464f57534e5ea, user_id=cf7bc4681c954c158a7051046bd9aa47, role_id=e1c88eebe911432cab6a07fbeb84653b)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:32:05.095392 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ce5940-798c-4820-a4fa-21a3445baa26 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:32:05.210412 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1234/2474] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jun 29 11:32:05 2022] PUT /identity/v3/projects/7e39c9a1ff3c46dcbe4464f57534e5ea/users/cf7bc4681c954c158a7051046bd9aa47/roles/e1c88eebe911432cab6a07fbeb84653b => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:32:05.217258 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40620f0c-f867-4ee8-8337-f2bd754278f6 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:32:05.217258 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40620f0c-f867-4ee8-8337-f2bd754278f6 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:32:05.217258 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40620f0c-f867-4ee8-8337-f2bd754278f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:32:05.234485 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-40620f0c-f867-4ee8-8337-f2bd754278f6 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:32:05.235710 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-40620f0c-f867-4ee8-8337-f2bd754278f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:32:05.301871 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1241/2475] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:32:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:32:05.312113 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ce5940-798c-4820-a4fa-21a3445baa26 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:32:05.313958 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-13ce5940-798c-4820-a4fa-21a3445baa26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:32:05.373640 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ce5940-798c-4820-a4fa-21a3445baa26 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:32:05.377226 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22de9031-ec26-45fe-97c8-9a8944f8802c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f4bb9f989e4024a5da28dced646b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:32:05.377932 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22de9031-ec26-45fe-97c8-9a8944f8802c service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:32:05.378183 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22de9031-ec26-45fe-97c8-9a8944f8802c service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:32:05.378451 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22de9031-ec26-45fe-97c8-9a8944f8802c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:32:05.387772 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22de9031-ec26-45fe-97c8-9a8944f8802c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:32:05.389555 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22de9031-ec26-45fe-97c8-9a8944f8802c service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:32:05.400911 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1235/2476] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:32:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:32:10.990870 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39561206-7011-473c-a222-8f4bfcb502a2 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:32:10.991344 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39561206-7011-473c-a222-8f4bfcb502a2 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:32:10.992249 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39561206-7011-473c-a222-8f4bfcb502a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:32:11.052094 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-39561206-7011-473c-a222-8f4bfcb502a2 None None] MFA Rules not processed for user `cf7bc4681c954c158a7051046bd9aa47`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:32:11.053615 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-39561206-7011-473c-a222-8f4bfcb502a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:32:11.140229 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:32:11.147889 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1242/2477] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:32:10 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 11:32:11.160775 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22de9031-ec26-45fe-97c8-9a8944f8802c service neutron] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:32:11.162345 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-22de9031-ec26-45fe-97c8-9a8944f8802c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:32:11.251885 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22de9031-ec26-45fe-97c8-9a8944f8802c service neutron] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:32:11.260563 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55a47a6-15e2-4718-bd72-e03f4bd4f721 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '796aeeff16df4739bddec1f6dee61bfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:32:11.261406 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55a47a6-15e2-4718-bd72-e03f4bd4f721 service nova] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:32:11.261690 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55a47a6-15e2-4718-bd72-e03f4bd4f721 service nova] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:32:11.261969 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55a47a6-15e2-4718-bd72-e03f4bd4f721 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:32:11.275949 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55a47a6-15e2-4718-bd72-e03f4bd4f721 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:32:11.277922 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55a47a6-15e2-4718-bd72-e03f4bd4f721 service nova] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:32:11.296097 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1236/2478] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:32:11 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:32:14.623041 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6b4cf6-05c0-4dd0-a16f-bd5382f18789 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:32:14.623930 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6b4cf6-05c0-4dd0-a16f-bd5382f18789 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:32:14.623930 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6b4cf6-05c0-4dd0-a16f-bd5382f18789 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:32:14.643285 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-3d6b4cf6-05c0-4dd0-a16f-bd5382f18789 None None] MFA Rules not processed for user `cf7bc4681c954c158a7051046bd9aa47`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:32:14.644576 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-3d6b4cf6-05c0-4dd0-a16f-bd5382f18789 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:32:14.652135 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1243/2479] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:32:14 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:32:21.284965 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=86,keystone:INSERT=2 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:32:23.964114 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4885c6-719b-41cf-bf03-e512b9aaac82 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:32:23.965283 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4885c6-719b-41cf-bf03-e512b9aaac82 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:32:23.965283 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4885c6-719b-41cf-bf03-e512b9aaac82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:32:23.984995 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-7d4885c6-719b-41cf-bf03-e512b9aaac82 None None] MFA Rules not processed for user `55487c2107a44c15b656b4865ee0d4fa`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:32:23.986391 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-7d4885c6-719b-41cf-bf03-e512b9aaac82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:32:24.051088 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1237/2480] 10.222.0.44 () {58 vars in 967 bytes} [Wed Jun 29 11:32:23 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:32:24.185233 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6b4cf6-05c0-4dd0-a16f-bd5382f18789 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:32:24.186686 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-3d6b4cf6-05c0-4dd0-a16f-bd5382f18789 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:32:24.259792 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6b4cf6-05c0-4dd0-a16f-bd5382f18789 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:32:24.265153 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f3582e-1a4e-4014-8db9-7df3dc16fa90 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3b38493ba904759badc800e396befda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:32:24.265812 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f3582e-1a4e-4014-8db9-7df3dc16fa90 service swift] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:32:24.266100 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f3582e-1a4e-4014-8db9-7df3dc16fa90 service swift] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:32:24.266378 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f3582e-1a4e-4014-8db9-7df3dc16fa90 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:32:24.275151 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f3582e-1a4e-4014-8db9-7df3dc16fa90 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:32:24.277441 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f3582e-1a4e-4014-8db9-7df3dc16fa90 service swift] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:32:24.286068 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1244/2481] 10.222.0.44 () {60 vars in 1384 bytes} [Wed Jun 29 11:32:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 29 11:32:26.394209 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469677c9-383b-4eb4-94b1-6eee163275dc None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:32:26.394713 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469677c9-383b-4eb4-94b1-6eee163275dc None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:32:26.394919 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469677c9-383b-4eb4-94b1-6eee163275dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:32:26.413763 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-469677c9-383b-4eb4-94b1-6eee163275dc None None] MFA Rules not processed for user `f75e0b3528b14856b73dc441c52d1d73`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:32:26.415093 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-469677c9-383b-4eb4-94b1-6eee163275dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:32:26.475473 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1238/2482] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:32:26 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:32:26.988935 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f3582e-1a4e-4014-8db9-7df3dc16fa90 service swift] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:32:27.000834 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f3582e-1a4e-4014-8db9-7df3dc16fa90 service swift] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:32:27.005197 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6528e3-a852-4bdd-8458-578846c5ec7b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f4bb9f989e4024a5da28dced646b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:32:27.006072 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6528e3-a852-4bdd-8458-578846c5ec7b service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:32:27.006513 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6528e3-a852-4bdd-8458-578846c5ec7b service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:32:27.006942 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6528e3-a852-4bdd-8458-578846c5ec7b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:32:27.011813 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-5c6528e3-a852-4bdd-8458-578846c5ec7b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:32:27.024537 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6528e3-a852-4bdd-8458-578846c5ec7b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:32:27.025994 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6528e3-a852-4bdd-8458-578846c5ec7b service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:32:27.036285 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1245/2483] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:32:26 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:32:27.916327 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469677c9-383b-4eb4-94b1-6eee163275dc None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:32:27.918750 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-469677c9-383b-4eb4-94b1-6eee163275dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:32:27.938220 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469677c9-383b-4eb4-94b1-6eee163275dc None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:32:27.944597 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb29baf-8c46-45d2-a54f-29920c85a434 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '796aeeff16df4739bddec1f6dee61bfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:32:27.946912 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb29baf-8c46-45d2-a54f-29920c85a434 service nova] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:32:27.947406 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb29baf-8c46-45d2-a54f-29920c85a434 service nova] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:32:27.947876 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb29baf-8c46-45d2-a54f-29920c85a434 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:32:27.950133 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-1fb29baf-8c46-45d2-a54f-29920c85a434 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:32:28.046793 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb29baf-8c46-45d2-a54f-29920c85a434 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:32:28.049292 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb29baf-8c46-45d2-a54f-29920c85a434 service nova] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:32:28.060883 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1239/2484] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:32:27 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:32:35.427976 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6528e3-a852-4bdd-8458-578846c5ec7b service neutron] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:32:35.429590 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-5c6528e3-a852-4bdd-8458-578846c5ec7b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:32:35.444125 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6528e3-a852-4bdd-8458-578846c5ec7b service neutron] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:32:35.449205 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769f48c1-a4b5-4390-a671-acb868939daf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:32:35.451738 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769f48c1-a4b5-4390-a671-acb868939daf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:32:35.451738 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769f48c1-a4b5-4390-a671-acb868939daf admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:32:35.451738 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769f48c1-a4b5-4390-a671-acb868939daf admin admin] PATH_INFO: `/v3/users/1158e0283f3643669448ca4281e7b1fa` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:32:35.452062 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769f48c1-a4b5-4390-a671-acb868939daf admin admin] RBAC: Authorizing `identity:delete_user(user_id=1158e0283f3643669448ca4281e7b1fa)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:32:35.453787 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769f48c1-a4b5-4390-a671-acb868939daf admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:32:35.689789 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-769f48c1-a4b5-4390-a671-acb868939daf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1158e0283f3643669448ca4281e7b1fa'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:32:35.690520 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-769f48c1-a4b5-4390-a671-acb868939daf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1158e0283f3643669448ca4281e7b1fa'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:32:35.804183 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-769f48c1-a4b5-4390-a671-acb868939daf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1158e0283f3643669448ca4281e7b1fa'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:32:35.813980 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-769f48c1-a4b5-4390-a671-acb868939daf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1158e0283f3643669448ca4281e7b1fa'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:32:35.815102 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-769f48c1-a4b5-4390-a671-acb868939daf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1158e0283f3643669448ca4281e7b1fa'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:32:35.841606 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1246/2485] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jun 29 11:32:35 2022] DELETE /identity/v3/users/1158e0283f3643669448ca4281e7b1fa => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:32:36.220239 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb29baf-8c46-45d2-a54f-29920c85a434 service nova] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:32:36.236200 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb29baf-8c46-45d2-a54f-29920c85a434 service nova] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:32:36.305911 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:32:36.324544 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ed083c-32ed-4b42-b3cb-02147e23af0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:32:36.326481 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ed083c-32ed-4b42-b3cb-02147e23af0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:32:36.326998 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ed083c-32ed-4b42-b3cb-02147e23af0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:32:36.327660 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ed083c-32ed-4b42-b3cb-02147e23af0f admin admin] PATH_INFO: `/v3/projects/58aac09bb66b4f98abf47ae8bd863203` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:32:36.329470 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ed083c-32ed-4b42-b3cb-02147e23af0f admin admin] RBAC: Authorizing `identity:delete_project(project_id=58aac09bb66b4f98abf47ae8bd863203)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:32:36.331474 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ed083c-32ed-4b42-b3cb-02147e23af0f admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:32:36.475656 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-c6ed083c-32ed-4b42-b3cb-02147e23af0f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '58aac09bb66b4f98abf47ae8bd863203', 'request_id': 'req-c6ed083c-32ed-4b42-b3cb-02147e23af0f'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:32:36.535251 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-c6ed083c-32ed-4b42-b3cb-02147e23af0f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '58aac09bb66b4f98abf47ae8bd863203', 'request_id': 'req-c6ed083c-32ed-4b42-b3cb-02147e23af0f'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:32:36.544505 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-c6ed083c-32ed-4b42-b3cb-02147e23af0f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '58aac09bb66b4f98abf47ae8bd863203', 'request_id': 'req-c6ed083c-32ed-4b42-b3cb-02147e23af0f'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:32:36.558683 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-c6ed083c-32ed-4b42-b3cb-02147e23af0f admin admin] The token cache is being invalidate because project 58aac09bb66b4f98abf47ae8bd863203 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 11:32:36.559189 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-c6ed083c-32ed-4b42-b3cb-02147e23af0f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:32:36.560086 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-c6ed083c-32ed-4b42-b3cb-02147e23af0f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:32:36.561809 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1240/2486] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jun 29 11:32:36 2022] DELETE /identity/v3/projects/58aac09bb66b4f98abf47ae8bd863203 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:32:36.565880 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769f48c1-a4b5-4390-a671-acb868939daf admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:32:36.567959 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-769f48c1-a4b5-4390-a671-acb868939daf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:32:36.581590 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769f48c1-a4b5-4390-a671-acb868939daf admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:32:36.642375 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:32:36.657802 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c947a59b-5f76-4772-9179-1081472c663f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:32:36.661613 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c947a59b-5f76-4772-9179-1081472c663f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:32:36.661613 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c947a59b-5f76-4772-9179-1081472c663f admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:32:36.661613 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c947a59b-5f76-4772-9179-1081472c663f admin admin] PATH_INFO: `/v3/users/82eaaa92763d40568351ef330bd68111` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:32:36.661613 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c947a59b-5f76-4772-9179-1081472c663f admin admin] RBAC: Authorizing `identity:delete_user(user_id=82eaaa92763d40568351ef330bd68111)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:32:36.663275 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c947a59b-5f76-4772-9179-1081472c663f admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:32:36.867969 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-c947a59b-5f76-4772-9179-1081472c663f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '82eaaa92763d40568351ef330bd68111'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:32:36.868639 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-c947a59b-5f76-4772-9179-1081472c663f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '82eaaa92763d40568351ef330bd68111'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:32:36.947475 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-c947a59b-5f76-4772-9179-1081472c663f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '82eaaa92763d40568351ef330bd68111'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:32:36.960685 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-c947a59b-5f76-4772-9179-1081472c663f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '82eaaa92763d40568351ef330bd68111'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:32:36.961704 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-c947a59b-5f76-4772-9179-1081472c663f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '82eaaa92763d40568351ef330bd68111'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:32:36.990027 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1247/2487] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jun 29 11:32:36 2022] DELETE /identity/v3/users/82eaaa92763d40568351ef330bd68111 => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:32:37.262923 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ed083c-32ed-4b42-b3cb-02147e23af0f admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:32:37.264977 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-c6ed083c-32ed-4b42-b3cb-02147e23af0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:32:37.285654 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ed083c-32ed-4b42-b3cb-02147e23af0f admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:32:37.376273 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f323eebc-3ffb-48db-b305-00448ab52d46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:32:37.378262 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f323eebc-3ffb-48db-b305-00448ab52d46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:32:37.378612 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f323eebc-3ffb-48db-b305-00448ab52d46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:32:37.379029 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f323eebc-3ffb-48db-b305-00448ab52d46 admin admin] PATH_INFO: `/v3/projects/d74b3636742f4d8aa46082cf1b0875f2` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:32:37.380537 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f323eebc-3ffb-48db-b305-00448ab52d46 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d74b3636742f4d8aa46082cf1b0875f2)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:32:37.383252 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f323eebc-3ffb-48db-b305-00448ab52d46 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:32:37.491147 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-f323eebc-3ffb-48db-b305-00448ab52d46 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd74b3636742f4d8aa46082cf1b0875f2', 'request_id': 'req-f323eebc-3ffb-48db-b305-00448ab52d46'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:32:37.563598 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-f323eebc-3ffb-48db-b305-00448ab52d46 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd74b3636742f4d8aa46082cf1b0875f2', 'request_id': 'req-f323eebc-3ffb-48db-b305-00448ab52d46'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:32:37.570542 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-f323eebc-3ffb-48db-b305-00448ab52d46 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd74b3636742f4d8aa46082cf1b0875f2', 'request_id': 'req-f323eebc-3ffb-48db-b305-00448ab52d46'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:32:37.583141 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-f323eebc-3ffb-48db-b305-00448ab52d46 admin admin] The token cache is being invalidate because project d74b3636742f4d8aa46082cf1b0875f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 11:32:37.583439 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-f323eebc-3ffb-48db-b305-00448ab52d46 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:32:37.584125 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-f323eebc-3ffb-48db-b305-00448ab52d46 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:32:37.585711 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1241/2488] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jun 29 11:32:37 2022] DELETE /identity/v3/projects/d74b3636742f4d8aa46082cf1b0875f2 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:32:41.462186 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb31a2f-fa46-45a8-84e7-9af45120b0fd None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:32:41.462681 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb31a2f-fa46-45a8-84e7-9af45120b0fd None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:32:41.462913 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb31a2f-fa46-45a8-84e7-9af45120b0fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:32:41.482744 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-1fb31a2f-fa46-45a8-84e7-9af45120b0fd None None] MFA Rules not processed for user `275df17696084dce9e7af487275bacab`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:32:41.483993 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-1fb31a2f-fa46-45a8-84e7-9af45120b0fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:32:41.536882 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1248/2489] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:32:41 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:32:47.580007 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=80,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:32:51.530271 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=59,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:33:43.161531 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f323eebc-3ffb-48db-b305-00448ab52d46 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:33:43.162942 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-f323eebc-3ffb-48db-b305-00448ab52d46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:33:43.231038 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f323eebc-3ffb-48db-b305-00448ab52d46 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:33:43.236622 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285c34bb-7165-474c-9f2b-1b029fcb7873 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a0505342b704d02977a5125957c2c9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:33:43.237263 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285c34bb-7165-474c-9f2b-1b029fcb7873 service placement] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:33:43.237545 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285c34bb-7165-474c-9f2b-1b029fcb7873 service placement] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:33:43.237818 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285c34bb-7165-474c-9f2b-1b029fcb7873 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:33:43.239553 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-285c34bb-7165-474c-9f2b-1b029fcb7873 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:33:43.251417 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285c34bb-7165-474c-9f2b-1b029fcb7873 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:33:43.253039 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285c34bb-7165-474c-9f2b-1b029fcb7873 service placement] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:33:43.263393 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1242/2490] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:33:43 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:33:53.258733 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=32 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:34:16.358529 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0600328d-45e1-45d2-b1b4-2a1bc4261e54 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:34:16.359039 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0600328d-45e1-45d2-b1b4-2a1bc4261e54 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:34:16.359152 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0600328d-45e1-45d2-b1b4-2a1bc4261e54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:34:16.376935 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-0600328d-45e1-45d2-b1b4-2a1bc4261e54 None None] MFA Rules not processed for user `cf7bc4681c954c158a7051046bd9aa47`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:34:16.380902 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-0600328d-45e1-45d2-b1b4-2a1bc4261e54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:34:16.431886 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1249/2491] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:34:16 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:34:26.325026 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285c34bb-7165-474c-9f2b-1b029fcb7873 service placement] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:34:26.326464 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-285c34bb-7165-474c-9f2b-1b029fcb7873 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:34:26.407220 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285c34bb-7165-474c-9f2b-1b029fcb7873 service placement] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:34:26.411092 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ead990d-c6b0-426d-a9ee-79d30d907cc4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff7931b75a924394b1af733febce0834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:34:26.412065 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ead990d-c6b0-426d-a9ee-79d30d907cc4 service cinder] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:34:26.413235 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ead990d-c6b0-426d-a9ee-79d30d907cc4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:34:26.413564 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ead990d-c6b0-426d-a9ee-79d30d907cc4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:34:26.416270 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-2ead990d-c6b0-426d-a9ee-79d30d907cc4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:34:26.424818 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=25 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:34:26.488639 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ead990d-c6b0-426d-a9ee-79d30d907cc4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:34:26.490261 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ead990d-c6b0-426d-a9ee-79d30d907cc4 service cinder] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:34:26.506530 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1243/2492] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:34:26 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:34:36.496918 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=48 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:34:40.149230 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71508ab0-f5b7-452d-80bd-2c8426acfe31 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:34:40.149743 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71508ab0-f5b7-452d-80bd-2c8426acfe31 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:34:40.149862 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71508ab0-f5b7-452d-80bd-2c8426acfe31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:34:40.167434 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-71508ab0-f5b7-452d-80bd-2c8426acfe31 None None] MFA Rules not processed for user `e91cc0b93f484f759233469ec5f37bb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:34:40.168669 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-71508ab0-f5b7-452d-80bd-2c8426acfe31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:34:40.175540 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1250/2493] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:34:40 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:34:50.161961 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=3 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:34:51.878414 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61c03cd-d864-4524-bec0-f8a65c8cac88 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:34:51.878943 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61c03cd-d864-4524-bec0-f8a65c8cac88 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:34:51.879056 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61c03cd-d864-4524-bec0-f8a65c8cac88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:34:51.896511 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-a61c03cd-d864-4524-bec0-f8a65c8cac88 None None] MFA Rules not processed for user `55487c2107a44c15b656b4865ee0d4fa`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:34:51.897737 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-a61c03cd-d864-4524-bec0-f8a65c8cac88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:34:51.975465 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1244/2494] 10.222.0.44 () {58 vars in 1009 bytes} [Wed Jun 29 11:34:51 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 11:34:52.159910 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71508ab0-f5b7-452d-80bd-2c8426acfe31 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:34:52.161371 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-71508ab0-f5b7-452d-80bd-2c8426acfe31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:34:52.234337 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71508ab0-f5b7-452d-80bd-2c8426acfe31 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:34:52.240799 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e68b171-a56c-4d24-b8ee-e778d5ac2871 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3b38493ba904759badc800e396befda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:34:52.241600 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e68b171-a56c-4d24-b8ee-e778d5ac2871 service swift] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:34:52.241824 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e68b171-a56c-4d24-b8ee-e778d5ac2871 service swift] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:34:52.242101 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e68b171-a56c-4d24-b8ee-e778d5ac2871 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:34:52.250655 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e68b171-a56c-4d24-b8ee-e778d5ac2871 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:34:52.252294 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e68b171-a56c-4d24-b8ee-e778d5ac2871 service swift] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:34:52.260430 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1251/2495] 10.222.0.44 () {60 vars in 1384 bytes} [Wed Jun 29 11:34:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 29 11:34:55.571928 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61c03cd-d864-4524-bec0-f8a65c8cac88 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:34:55.573443 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-a61c03cd-d864-4524-bec0-f8a65c8cac88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:34:55.635953 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61c03cd-d864-4524-bec0-f8a65c8cac88 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:34:55.640060 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186a504f-170f-4bee-8f2e-4c48cff6110d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f4bb9f989e4024a5da28dced646b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:34:55.640706 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186a504f-170f-4bee-8f2e-4c48cff6110d service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:34:55.640986 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186a504f-170f-4bee-8f2e-4c48cff6110d service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:34:55.641260 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186a504f-170f-4bee-8f2e-4c48cff6110d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:34:55.643007 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-186a504f-170f-4bee-8f2e-4c48cff6110d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:34:55.718287 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186a504f-170f-4bee-8f2e-4c48cff6110d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:34:55.719719 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186a504f-170f-4bee-8f2e-4c48cff6110d service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:34:55.732159 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1245/2496] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:34:55 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:34:59.742041 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e68b171-a56c-4d24-b8ee-e778d5ac2871 service swift] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:34:59.743525 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-9e68b171-a56c-4d24-b8ee-e778d5ac2871 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:34:59.755334 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e68b171-a56c-4d24-b8ee-e778d5ac2871 service swift] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:34:59.758953 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134feed0-4462-433e-837b-f4b272006618 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f4bb9f989e4024a5da28dced646b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:34:59.759584 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134feed0-4462-433e-837b-f4b272006618 service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:34:59.759898 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134feed0-4462-433e-837b-f4b272006618 service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:34:59.760178 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134feed0-4462-433e-837b-f4b272006618 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:34:59.761886 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-134feed0-4462-433e-837b-f4b272006618 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:34:59.827734 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134feed0-4462-433e-837b-f4b272006618 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:34:59.829205 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134feed0-4462-433e-837b-f4b272006618 service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:34:59.841537 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1252/2497] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:34:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:35:01.811886 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186a504f-170f-4bee-8f2e-4c48cff6110d service neutron] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:35:01.823317 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186a504f-170f-4bee-8f2e-4c48cff6110d service neutron] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:35:01.827536 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a217c9dd-6a53-4989-aa0f-1b21eba2240d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f4bb9f989e4024a5da28dced646b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:35:01.828374 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a217c9dd-6a53-4989-aa0f-1b21eba2240d service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:35:01.828670 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a217c9dd-6a53-4989-aa0f-1b21eba2240d service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:35:01.828964 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a217c9dd-6a53-4989-aa0f-1b21eba2240d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:35:01.831072 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-a217c9dd-6a53-4989-aa0f-1b21eba2240d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:35:01.950260 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:35:01.989597 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a217c9dd-6a53-4989-aa0f-1b21eba2240d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:35:01.993299 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a217c9dd-6a53-4989-aa0f-1b21eba2240d service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:35:02.003897 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1246/2498] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:35:01 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:35:04.270023 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134feed0-4462-433e-837b-f4b272006618 service neutron] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:35:04.271494 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-134feed0-4462-433e-837b-f4b272006618 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:35:04.288304 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134feed0-4462-433e-837b-f4b272006618 service neutron] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:35:04.294200 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d645edd-7117-4972-ac1f-a91363e45315 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a0505342b704d02977a5125957c2c9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:35:04.295157 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d645edd-7117-4972-ac1f-a91363e45315 service placement] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:35:04.295781 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d645edd-7117-4972-ac1f-a91363e45315 service placement] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:35:04.296842 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d645edd-7117-4972-ac1f-a91363e45315 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:35:04.299008 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-0d645edd-7117-4972-ac1f-a91363e45315 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:35:04.312409 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d645edd-7117-4972-ac1f-a91363e45315 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:35:04.314575 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d645edd-7117-4972-ac1f-a91363e45315 service placement] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:35:04.328475 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1253/2499] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:35:04 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:35:04.461374 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a217c9dd-6a53-4989-aa0f-1b21eba2240d service neutron] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:35:04.465273 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-a217c9dd-6a53-4989-aa0f-1b21eba2240d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:35:04.495081 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a217c9dd-6a53-4989-aa0f-1b21eba2240d service neutron] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:35:04.506373 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e61166-307b-4f78-81c7-3512b88032c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:35:04.507662 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e61166-307b-4f78-81c7-3512b88032c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:35:04.508156 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e61166-307b-4f78-81c7-3512b88032c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:35:04.508588 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e61166-307b-4f78-81c7-3512b88032c5 admin admin] PATH_INFO: `/v3/users/f75e0b3528b14856b73dc441c52d1d73` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:35:04.529187 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e61166-307b-4f78-81c7-3512b88032c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f75e0b3528b14856b73dc441c52d1d73)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:35:04.531510 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e61166-307b-4f78-81c7-3512b88032c5 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:35:04.767538 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-75e61166-307b-4f78-81c7-3512b88032c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f75e0b3528b14856b73dc441c52d1d73'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:35:04.847005 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-75e61166-307b-4f78-81c7-3512b88032c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f75e0b3528b14856b73dc441c52d1d73'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:35:04.848004 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-75e61166-307b-4f78-81c7-3512b88032c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f75e0b3528b14856b73dc441c52d1d73'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:35:04.863750 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-75e61166-307b-4f78-81c7-3512b88032c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f75e0b3528b14856b73dc441c52d1d73'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:35:04.864726 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-75e61166-307b-4f78-81c7-3512b88032c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f75e0b3528b14856b73dc441c52d1d73'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:35:04.892607 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1247/2500] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jun 29 11:35:04 2022] DELETE /identity/v3/users/f75e0b3528b14856b73dc441c52d1d73 => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:35:05.339157 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d645edd-7117-4972-ac1f-a91363e45315 service placement] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:35:05.357283 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d645edd-7117-4972-ac1f-a91363e45315 service placement] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:35:05.437806 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc8be93-0b26-4a36-8561-beb8a6b25867 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:35:05.438686 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc8be93-0b26-4a36-8561-beb8a6b25867 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:35:05.438971 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc8be93-0b26-4a36-8561-beb8a6b25867 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:35:05.439248 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc8be93-0b26-4a36-8561-beb8a6b25867 admin admin] PATH_INFO: `/v3/projects/2c08a3d7e74f45999d1ba0ab11781458` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:35:05.452847 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc8be93-0b26-4a36-8561-beb8a6b25867 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c08a3d7e74f45999d1ba0ab11781458)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:35:05.454847 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc8be93-0b26-4a36-8561-beb8a6b25867 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:35:05.457549 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:35:05.592043 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-cdc8be93-0b26-4a36-8561-beb8a6b25867 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c08a3d7e74f45999d1ba0ab11781458', 'request_id': 'req-cdc8be93-0b26-4a36-8561-beb8a6b25867'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:35:05.598927 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-cdc8be93-0b26-4a36-8561-beb8a6b25867 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c08a3d7e74f45999d1ba0ab11781458', 'request_id': 'req-cdc8be93-0b26-4a36-8561-beb8a6b25867'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:35:05.604661 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-cdc8be93-0b26-4a36-8561-beb8a6b25867 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c08a3d7e74f45999d1ba0ab11781458', 'request_id': 'req-cdc8be93-0b26-4a36-8561-beb8a6b25867'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:35:05.694681 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-cdc8be93-0b26-4a36-8561-beb8a6b25867 admin admin] The token cache is being invalidate because project 2c08a3d7e74f45999d1ba0ab11781458 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 11:35:05.695049 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-cdc8be93-0b26-4a36-8561-beb8a6b25867 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:35:05.695705 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-cdc8be93-0b26-4a36-8561-beb8a6b25867 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:35:05.697862 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1254/2501] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jun 29 11:35:05 2022] DELETE /identity/v3/projects/2c08a3d7e74f45999d1ba0ab11781458 => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:35:05.703575 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2411f0ea-6171-4ed4-90ab-1bd7e751440f None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:35:05.704117 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2411f0ea-6171-4ed4-90ab-1bd7e751440f None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:35:05.704214 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2411f0ea-6171-4ed4-90ab-1bd7e751440f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:35:05.726314 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-2411f0ea-6171-4ed4-90ab-1bd7e751440f None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:35:05.727606 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-2411f0ea-6171-4ed4-90ab-1bd7e751440f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:35:05.791341 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1248/2502] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:35:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:35:05.799733 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b27ddcd-d095-434a-a221-0ea7c5b50213 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:35:05.800164 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b27ddcd-d095-434a-a221-0ea7c5b50213 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:35:05.800630 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b27ddcd-d095-434a-a221-0ea7c5b50213 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:35:05.820088 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-8b27ddcd-d095-434a-a221-0ea7c5b50213 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:35:05.821349 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-8b27ddcd-d095-434a-a221-0ea7c5b50213 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:35:05.829854 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1255/2503] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:35:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:35:05.833882 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2411f0ea-6171-4ed4-90ab-1bd7e751440f None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:35:05.845229 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2411f0ea-6171-4ed4-90ab-1bd7e751440f None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:35:05.850025 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05c8743-963c-4535-8c7f-983a09258e90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:35:05.850689 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05c8743-963c-4535-8c7f-983a09258e90 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:35:05.850980 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05c8743-963c-4535-8c7f-983a09258e90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:35:05.851253 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05c8743-963c-4535-8c7f-983a09258e90 admin admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:35:05.852030 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05c8743-963c-4535-8c7f-983a09258e90 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:35:05.852343 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05c8743-963c-4535-8c7f-983a09258e90 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:35:05.854239 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05c8743-963c-4535-8c7f-983a09258e90 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:35:05.867567 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1249/2504] 10.222.0.44 () {58 vars in 1152 bytes} [Wed Jun 29 11:35:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:35:05.880267 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b27ddcd-d095-434a-a221-0ea7c5b50213 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:35:05.890165 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b27ddcd-d095-434a-a221-0ea7c5b50213 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:35:05.895823 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff52a68-8f69-4cc8-8bcf-501a7b7d7dc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:35:05.896518 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff52a68-8f69-4cc8-8bcf-501a7b7d7dc9 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:35:05.896802 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff52a68-8f69-4cc8-8bcf-501a7b7d7dc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:35:05.897075 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff52a68-8f69-4cc8-8bcf-501a7b7d7dc9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:35:05.897767 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff52a68-8f69-4cc8-8bcf-501a7b7d7dc9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:35:05.899770 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff52a68-8f69-4cc8-8bcf-501a7b7d7dc9 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:35:06.011950 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1256/2505] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jun 29 11:35:05 2022] POST /identity/v3/projects => generated 371 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:35:06.016223 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05c8743-963c-4535-8c7f-983a09258e90 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:35:06.028361 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05c8743-963c-4535-8c7f-983a09258e90 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:35:06.061955 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:35:06.094843 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f4ee00-91fe-4f86-afd9-e1fff5ddc326 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:35:06.095642 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f4ee00-91fe-4f86-afd9-e1fff5ddc326 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:35:06.095963 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f4ee00-91fe-4f86-afd9-e1fff5ddc326 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:35:06.096231 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f4ee00-91fe-4f86-afd9-e1fff5ddc326 admin admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:35:06.096936 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f4ee00-91fe-4f86-afd9-e1fff5ddc326 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:35:06.098916 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f4ee00-91fe-4f86-afd9-e1fff5ddc326 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:35:06.203553 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1250/2506] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jun 29 11:35:06 2022] POST /identity/v3/users => generated 286 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:35:06.204595 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff52a68-8f69-4cc8-8bcf-501a7b7d7dc9 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:35:06.218219 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff52a68-8f69-4cc8-8bcf-501a7b7d7dc9 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:35:06.223789 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824277b6-6276-445f-ba5c-f39fbea22a03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:35:06.224650 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824277b6-6276-445f-ba5c-f39fbea22a03 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:35:06.225063 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824277b6-6276-445f-ba5c-f39fbea22a03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:35:06.225463 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824277b6-6276-445f-ba5c-f39fbea22a03 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:35:06.226324 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824277b6-6276-445f-ba5c-f39fbea22a03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:35:06.228190 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824277b6-6276-445f-ba5c-f39fbea22a03 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:35:06.242094 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1257/2507] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:35:06 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 11:35:06.245940 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f4ee00-91fe-4f86-afd9-e1fff5ddc326 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:35:06.257023 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f4ee00-91fe-4f86-afd9-e1fff5ddc326 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:35:06.261887 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acba0e7-3e41-4060-8993-e6589f960006 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:35:06.262569 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acba0e7-3e41-4060-8993-e6589f960006 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:35:06.262851 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acba0e7-3e41-4060-8993-e6589f960006 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:35:06.263131 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acba0e7-3e41-4060-8993-e6589f960006 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:35:06.263874 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acba0e7-3e41-4060-8993-e6589f960006 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:35:06.265501 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acba0e7-3e41-4060-8993-e6589f960006 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:35:06.276593 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1251/2508] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:35:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 11:35:06.281449 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824277b6-6276-445f-ba5c-f39fbea22a03 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:35:06.292247 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824277b6-6276-445f-ba5c-f39fbea22a03 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:35:06.297840 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c127b38-5abf-4a49-94f6-ade942f7e404 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:35:06.298740 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c127b38-5abf-4a49-94f6-ade942f7e404 admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:35:06.299026 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c127b38-5abf-4a49-94f6-ade942f7e404 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:35:06.299302 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c127b38-5abf-4a49-94f6-ade942f7e404 admin admin] PATH_INFO: `/v3/projects/49d52997dabc45fcb997fd4b6fa2a80e/users/05d3d290d96a444a99406eaa51dbd3a0/roles/e1c88eebe911432cab6a07fbeb84653b` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:35:06.315355 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c127b38-5abf-4a49-94f6-ade942f7e404 admin admin] RBAC: Authorizing `identity:create_grant(project_id=49d52997dabc45fcb997fd4b6fa2a80e, user_id=05d3d290d96a444a99406eaa51dbd3a0, role_id=e1c88eebe911432cab6a07fbeb84653b)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:35:06.318051 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c127b38-5abf-4a49-94f6-ade942f7e404 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:35:06.412997 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1258/2509] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jun 29 11:35:06 2022] PUT /identity/v3/projects/49d52997dabc45fcb997fd4b6fa2a80e/users/05d3d290d96a444a99406eaa51dbd3a0/roles/e1c88eebe911432cab6a07fbeb84653b => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:35:06.428775 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09682c0-6359-4886-9754-6541057db50e None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:35:06.429096 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09682c0-6359-4886-9754-6541057db50e None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:35:06.429375 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09682c0-6359-4886-9754-6541057db50e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:35:06.457999 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-c09682c0-6359-4886-9754-6541057db50e None None] MFA Rules not processed for user `05d3d290d96a444a99406eaa51dbd3a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:35:06.459281 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-c09682c0-6359-4886-9754-6541057db50e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:35:06.528570 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1252/2510] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:35:06 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:35:06.540474 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c127b38-5abf-4a49-94f6-ade942f7e404 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:35:06.542211 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-5c127b38-5abf-4a49-94f6-ade942f7e404 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:35:06.618614 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c127b38-5abf-4a49-94f6-ade942f7e404 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:35:06.624986 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c583ac-87ff-428c-b5fa-ad8701e75a4a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '796aeeff16df4739bddec1f6dee61bfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:35:06.625702 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c583ac-87ff-428c-b5fa-ad8701e75a4a service nova] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:35:06.625909 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c583ac-87ff-428c-b5fa-ad8701e75a4a service nova] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:35:06.626187 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c583ac-87ff-428c-b5fa-ad8701e75a4a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:35:06.634299 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c583ac-87ff-428c-b5fa-ad8701e75a4a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:35:06.635897 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c583ac-87ff-428c-b5fa-ad8701e75a4a service nova] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:35:06.645366 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1259/2511] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:35:06 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:35:06.666628 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09682c0-6359-4886-9754-6541057db50e None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:35:06.668045 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-c09682c0-6359-4886-9754-6541057db50e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:35:06.731743 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09682c0-6359-4886-9754-6541057db50e None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:35:06.739074 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcf2fc9-a2f8-4b9e-8771-d5d90147a2c5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff7931b75a924394b1af733febce0834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:35:06.740426 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcf2fc9-a2f8-4b9e-8771-d5d90147a2c5 service cinder] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:35:06.740710 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcf2fc9-a2f8-4b9e-8771-d5d90147a2c5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:35:06.740984 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcf2fc9-a2f8-4b9e-8771-d5d90147a2c5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:35:06.742893 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-5fcf2fc9-a2f8-4b9e-8771-d5d90147a2c5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:35:06.761505 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcf2fc9-a2f8-4b9e-8771-d5d90147a2c5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:35:06.764112 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcf2fc9-a2f8-4b9e-8771-d5d90147a2c5 service cinder] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:35:06.778715 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1253/2512] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:35:06 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:35:09.948621 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c583ac-87ff-428c-b5fa-ad8701e75a4a service nova] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:35:09.960814 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:35:10.013791 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c583ac-87ff-428c-b5fa-ad8701e75a4a service nova] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:35:10.018143 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561c0726-8c66-46b5-b1df-084305440a7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:35:10.019008 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561c0726-8c66-46b5-b1df-084305440a7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:35:10.019285 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561c0726-8c66-46b5-b1df-084305440a7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:35:10.019557 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561c0726-8c66-46b5-b1df-084305440a7a admin admin] PATH_INFO: `/v3/users/05d3d290d96a444a99406eaa51dbd3a0` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:35:10.020759 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561c0726-8c66-46b5-b1df-084305440a7a admin admin] RBAC: Authorizing `identity:delete_user(user_id=05d3d290d96a444a99406eaa51dbd3a0)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:35:10.022644 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561c0726-8c66-46b5-b1df-084305440a7a admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:35:10.218837 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-561c0726-8c66-46b5-b1df-084305440a7a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '05d3d290d96a444a99406eaa51dbd3a0'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:35:10.219481 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-561c0726-8c66-46b5-b1df-084305440a7a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '05d3d290d96a444a99406eaa51dbd3a0'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:35:10.280110 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-561c0726-8c66-46b5-b1df-084305440a7a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '05d3d290d96a444a99406eaa51dbd3a0'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:35:10.287247 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-561c0726-8c66-46b5-b1df-084305440a7a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '05d3d290d96a444a99406eaa51dbd3a0'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:35:10.287904 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-561c0726-8c66-46b5-b1df-084305440a7a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '05d3d290d96a444a99406eaa51dbd3a0'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:35:10.306261 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1260/2513] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jun 29 11:35:09 2022] DELETE /identity/v3/users/05d3d290d96a444a99406eaa51dbd3a0 => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:35:10.311575 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a29da99-d051-47ca-9f1e-bfa786f36940 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:35:10.312064 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a29da99-d051-47ca-9f1e-bfa786f36940 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:35:10.313022 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a29da99-d051-47ca-9f1e-bfa786f36940 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:35:10.329615 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-3a29da99-d051-47ca-9f1e-bfa786f36940 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:35:10.331016 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-3a29da99-d051-47ca-9f1e-bfa786f36940 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:35:10.365615 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:35:10.390017 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1254/2514] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:35:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:35:10.399635 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561c0726-8c66-46b5-b1df-084305440a7a admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:35:10.401074 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-561c0726-8c66-46b5-b1df-084305440a7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:35:10.459891 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561c0726-8c66-46b5-b1df-084305440a7a admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:35:10.463532 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cb3020-326c-4b00-a56b-e65fd323ea24 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f4bb9f989e4024a5da28dced646b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:35:10.464219 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cb3020-326c-4b00-a56b-e65fd323ea24 service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:35:10.464505 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cb3020-326c-4b00-a56b-e65fd323ea24 service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:35:10.464805 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cb3020-326c-4b00-a56b-e65fd323ea24 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:35:10.474239 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cb3020-326c-4b00-a56b-e65fd323ea24 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:35:10.475703 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cb3020-326c-4b00-a56b-e65fd323ea24 service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:35:10.486118 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1261/2515] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:35:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:35:10.894286 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a29da99-d051-47ca-9f1e-bfa786f36940 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:35:10.894286 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-3a29da99-d051-47ca-9f1e-bfa786f36940 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:35:10.908802 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a29da99-d051-47ca-9f1e-bfa786f36940 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:35:10.914307 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6849f4b-644d-4a6c-b501-512c1b24a762 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:35:10.915195 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6849f4b-644d-4a6c-b501-512c1b24a762 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:35:10.915478 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6849f4b-644d-4a6c-b501-512c1b24a762 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:35:10.915945 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6849f4b-644d-4a6c-b501-512c1b24a762 admin admin] PATH_INFO: `/v3/projects/49d52997dabc45fcb997fd4b6fa2a80e` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:35:10.917860 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6849f4b-644d-4a6c-b501-512c1b24a762 admin admin] RBAC: Authorizing `identity:delete_project(project_id=49d52997dabc45fcb997fd4b6fa2a80e)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:35:10.919822 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6849f4b-644d-4a6c-b501-512c1b24a762 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:35:11.074599 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-b6849f4b-644d-4a6c-b501-512c1b24a762 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '49d52997dabc45fcb997fd4b6fa2a80e', 'request_id': 'req-b6849f4b-644d-4a6c-b501-512c1b24a762'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:35:11.168913 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-b6849f4b-644d-4a6c-b501-512c1b24a762 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '49d52997dabc45fcb997fd4b6fa2a80e', 'request_id': 'req-b6849f4b-644d-4a6c-b501-512c1b24a762'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:35:11.175705 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-b6849f4b-644d-4a6c-b501-512c1b24a762 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '49d52997dabc45fcb997fd4b6fa2a80e', 'request_id': 'req-b6849f4b-644d-4a6c-b501-512c1b24a762'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:35:11.187058 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-b6849f4b-644d-4a6c-b501-512c1b24a762 admin admin] The token cache is being invalidate because project 49d52997dabc45fcb997fd4b6fa2a80e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 11:35:11.187350 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-b6849f4b-644d-4a6c-b501-512c1b24a762 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:35:11.188046 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-b6849f4b-644d-4a6c-b501-512c1b24a762 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:35:11.189839 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1255/2516] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jun 29 11:35:10 2022] DELETE /identity/v3/projects/49d52997dabc45fcb997fd4b6fa2a80e => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:35:11.194869 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc0961e-dda0-4c26-bd3e-1fedd7c61163 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:35:11.195362 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc0961e-dda0-4c26-bd3e-1fedd7c61163 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:35:11.195461 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc0961e-dda0-4c26-bd3e-1fedd7c61163 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:35:11.213212 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-5bc0961e-dda0-4c26-bd3e-1fedd7c61163 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:35:11.214455 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-5bc0961e-dda0-4c26-bd3e-1fedd7c61163 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:35:11.245300 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:35:11.274954 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1262/2517] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:35:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:35:11.283887 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12746ab7-ae5a-4c9a-8d9e-2b70facb3a62 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:35:11.284187 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12746ab7-ae5a-4c9a-8d9e-2b70facb3a62 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:35:11.284471 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12746ab7-ae5a-4c9a-8d9e-2b70facb3a62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:35:11.302858 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-12746ab7-ae5a-4c9a-8d9e-2b70facb3a62 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:35:11.304129 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-12746ab7-ae5a-4c9a-8d9e-2b70facb3a62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:35:11.318951 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1256/2518] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:35:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:35:11.323423 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc0961e-dda0-4c26-bd3e-1fedd7c61163 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:35:11.334916 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc0961e-dda0-4c26-bd3e-1fedd7c61163 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:35:11.340677 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f39b6e-f056-4766-84ee-e8ccc0b75974 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:35:11.341353 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f39b6e-f056-4766-84ee-e8ccc0b75974 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:35:11.341636 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f39b6e-f056-4766-84ee-e8ccc0b75974 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:35:11.341910 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f39b6e-f056-4766-84ee-e8ccc0b75974 admin admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:35:11.342639 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f39b6e-f056-4766-84ee-e8ccc0b75974 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:35:11.342947 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f39b6e-f056-4766-84ee-e8ccc0b75974 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:35:11.344717 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f39b6e-f056-4766-84ee-e8ccc0b75974 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:35:11.357443 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1263/2519] 10.222.0.44 () {58 vars in 1152 bytes} [Wed Jun 29 11:35:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:35:11.361836 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12746ab7-ae5a-4c9a-8d9e-2b70facb3a62 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:35:11.371147 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12746ab7-ae5a-4c9a-8d9e-2b70facb3a62 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:35:11.376363 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dff99fb-6370-42a5-9706-c638181f9a78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:35:11.377013 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dff99fb-6370-42a5-9706-c638181f9a78 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:35:11.377297 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dff99fb-6370-42a5-9706-c638181f9a78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:35:11.377572 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dff99fb-6370-42a5-9706-c638181f9a78 admin admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:35:11.378386 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dff99fb-6370-42a5-9706-c638181f9a78 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:35:11.380334 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dff99fb-6370-42a5-9706-c638181f9a78 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:35:11.508341 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1257/2520] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jun 29 11:35:11 2022] POST /identity/v3/projects => generated 363 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:35:11.509535 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f39b6e-f056-4766-84ee-e8ccc0b75974 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:35:11.519472 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f39b6e-f056-4766-84ee-e8ccc0b75974 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:35:11.576222 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e247cd-86f9-4345-89de-18278cf9c84b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:35:11.576887 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e247cd-86f9-4345-89de-18278cf9c84b admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:35:11.577181 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e247cd-86f9-4345-89de-18278cf9c84b admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:35:11.577447 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e247cd-86f9-4345-89de-18278cf9c84b admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:35:11.578123 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e247cd-86f9-4345-89de-18278cf9c84b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:35:11.580110 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e247cd-86f9-4345-89de-18278cf9c84b admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:35:11.687564 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1264/2521] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jun 29 11:35:11 2022] POST /identity/v3/users => generated 282 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:35:11.691460 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dff99fb-6370-42a5-9706-c638181f9a78 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:35:11.701914 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dff99fb-6370-42a5-9706-c638181f9a78 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:35:11.706437 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa98d3a8-6546-4d7e-b825-a89cd0d7bb9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:35:11.707081 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa98d3a8-6546-4d7e-b825-a89cd0d7bb9b admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:35:11.707367 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa98d3a8-6546-4d7e-b825-a89cd0d7bb9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:35:11.707637 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa98d3a8-6546-4d7e-b825-a89cd0d7bb9b admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:35:11.708410 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa98d3a8-6546-4d7e-b825-a89cd0d7bb9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:35:11.712866 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa98d3a8-6546-4d7e-b825-a89cd0d7bb9b admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:35:11.723360 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1258/2522] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:35:11 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 11:35:11.724719 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e247cd-86f9-4345-89de-18278cf9c84b admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:35:11.737128 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e247cd-86f9-4345-89de-18278cf9c84b admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:35:11.742941 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48254a3-416f-449a-baef-366921af974b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:35:11.743680 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48254a3-416f-449a-baef-366921af974b admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:35:11.746153 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48254a3-416f-449a-baef-366921af974b admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:35:11.746153 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48254a3-416f-449a-baef-366921af974b admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:35:11.746153 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48254a3-416f-449a-baef-366921af974b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:35:11.746975 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48254a3-416f-449a-baef-366921af974b admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:35:11.758673 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1265/2523] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:35:11 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 11:35:11.762909 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa98d3a8-6546-4d7e-b825-a89cd0d7bb9b admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:35:11.773155 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa98d3a8-6546-4d7e-b825-a89cd0d7bb9b admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:35:11.778041 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a415ab28-7851-4e1a-92da-67a45df1d032 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:35:11.778944 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a415ab28-7851-4e1a-92da-67a45df1d032 admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:35:11.779224 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a415ab28-7851-4e1a-92da-67a45df1d032 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:35:11.779497 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a415ab28-7851-4e1a-92da-67a45df1d032 admin admin] PATH_INFO: `/v3/projects/5de11f6ce85a49ed91cc17930759e641/users/5fce995667af44c890cbb5d4d52d67cb/roles/e1c88eebe911432cab6a07fbeb84653b` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:35:11.796716 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a415ab28-7851-4e1a-92da-67a45df1d032 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5de11f6ce85a49ed91cc17930759e641, user_id=5fce995667af44c890cbb5d4d52d67cb, role_id=e1c88eebe911432cab6a07fbeb84653b)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:35:11.799731 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a415ab28-7851-4e1a-92da-67a45df1d032 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:35:11.826157 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44f0799-dfff-45da-9dff-503f2483cd4f None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:35:11.826523 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44f0799-dfff-45da-9dff-503f2483cd4f None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:35:11.826878 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44f0799-dfff-45da-9dff-503f2483cd4f None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:35:11.828403 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1266/2524] 10.222.0.44 () {54 vars in 921 bytes} [Wed Jun 29 11:35:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 29 11:35:11.833693 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce1446b-fa73-4ce1-96e1-bdbb4eb296e2 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:35:11.833968 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce1446b-fa73-4ce1-96e1-bdbb4eb296e2 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:35:11.834245 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce1446b-fa73-4ce1-96e1-bdbb4eb296e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:35:11.853862 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-1ce1446b-fa73-4ce1-96e1-bdbb4eb296e2 None None] MFA Rules not processed for user `796aeeff16df4739bddec1f6dee61bfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:35:11.855082 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-1ce1446b-fa73-4ce1-96e1-bdbb4eb296e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:35:11.919803 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1259/2525] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jun 29 11:35:11 2022] PUT /identity/v3/projects/5de11f6ce85a49ed91cc17930759e641/users/5fce995667af44c890cbb5d4d52d67cb/roles/e1c88eebe911432cab6a07fbeb84653b => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:35:11.928546 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bda891d-009f-44c2-9d2f-385cf59cc0bd None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:35:11.928835 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bda891d-009f-44c2-9d2f-385cf59cc0bd None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:35:11.929109 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bda891d-009f-44c2-9d2f-385cf59cc0bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:35:11.937241 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1267/2526] 10.222.0.44 () {58 vars in 1018 bytes} [Wed Jun 29 11:35:11 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:35:11.948984 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce1446b-fa73-4ce1-96e1-bdbb4eb296e2 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:35:11.950547 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-1ce1446b-fa73-4ce1-96e1-bdbb4eb296e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:35:11.956017 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-4bda891d-009f-44c2-9d2f-385cf59cc0bd None None] MFA Rules not processed for user `5fce995667af44c890cbb5d4d52d67cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:35:11.957206 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-4bda891d-009f-44c2-9d2f-385cf59cc0bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:35:11.987980 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:35:12.004621 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:35:12.012168 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce1446b-fa73-4ce1-96e1-bdbb4eb296e2 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:35:12.022335 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1260/2527] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:35:11 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 11:35:12.026371 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bda891d-009f-44c2-9d2f-385cf59cc0bd None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:35:12.061058 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccaf475a-c902-40c5-b3c6-69bb44dc9381 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede3a17c7063432e8ee66462766584f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:35:12.061705 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaf475a-c902-40c5-b3c6-69bb44dc9381 service glance] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:35:12.061987 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaf475a-c902-40c5-b3c6-69bb44dc9381 service glance] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:35:12.062263 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaf475a-c902-40c5-b3c6-69bb44dc9381 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:35:12.085960 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bda891d-009f-44c2-9d2f-385cf59cc0bd None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:35:12.090442 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271dc332-71f1-4589-9ffe-3dbe6a5d313d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:35:12.091616 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271dc332-71f1-4589-9ffe-3dbe6a5d313d admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:35:12.091941 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271dc332-71f1-4589-9ffe-3dbe6a5d313d admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:35:12.092223 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271dc332-71f1-4589-9ffe-3dbe6a5d313d admin admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:35:12.092907 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271dc332-71f1-4589-9ffe-3dbe6a5d313d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:35:12.094814 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271dc332-71f1-4589-9ffe-3dbe6a5d313d admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:35:12.128905 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccaf475a-c902-40c5-b3c6-69bb44dc9381 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:35:12.130263 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccaf475a-c902-40c5-b3c6-69bb44dc9381 service glance] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:35:12.141280 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1268/2528] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:35:11 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:35:12.190935 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1261/2529] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jun 29 11:35:12 2022] POST /identity/v3/projects => generated 365 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 11:35:12.195028 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccaf475a-c902-40c5-b3c6-69bb44dc9381 service glance] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:35:12.206022 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccaf475a-c902-40c5-b3c6-69bb44dc9381 service glance] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:35:12.263213 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ececae-bdb7-4c05-a336-aaec21fb3070 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:35:12.263914 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ececae-bdb7-4c05-a336-aaec21fb3070 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:35:12.264273 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ececae-bdb7-4c05-a336-aaec21fb3070 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:35:12.264551 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ececae-bdb7-4c05-a336-aaec21fb3070 admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:35:12.265241 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ececae-bdb7-4c05-a336-aaec21fb3070 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:35:12.267149 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ececae-bdb7-4c05-a336-aaec21fb3070 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:35:12.334664 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e4ee5b-5019-4f11-b6b5-0682c9f30a5a None None] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:35:12.335190 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e4ee5b-5019-4f11-b6b5-0682c9f30a5a None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:35:12.335305 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e4ee5b-5019-4f11-b6b5-0682c9f30a5a None None] PATH_INFO: `/` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:35:12.336631 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1262/2530] 10.222.0.44 () {54 vars in 921 bytes} [Wed Jun 29 11:35:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 29 11:35:12.341544 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2a47ba-1526-4a00-84a8-20171fd99138 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:35:12.341873 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2a47ba-1526-4a00-84a8-20171fd99138 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:35:12.342147 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2a47ba-1526-4a00-84a8-20171fd99138 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:35:12.363263 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1269/2531] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jun 29 11:35:12 2022] POST /identity/v3/users => generated 289 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:35:12.366985 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ececae-bdb7-4c05-a336-aaec21fb3070 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:35:12.377842 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ececae-bdb7-4c05-a336-aaec21fb3070 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:35:12.382968 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0914a08b-df47-4f49-aa7d-a8125a2b2c9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:35:12.383607 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0914a08b-df47-4f49-aa7d-a8125a2b2c9b admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:35:12.384017 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0914a08b-df47-4f49-aa7d-a8125a2b2c9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:35:12.388808 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0914a08b-df47-4f49-aa7d-a8125a2b2c9b admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:35:12.388808 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0914a08b-df47-4f49-aa7d-a8125a2b2c9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:35:12.389017 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-ed2a47ba-1526-4a00-84a8-20171fd99138 None None] MFA Rules not processed for user `b5f4bb9f989e4024a5da28dced646b92`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:35:12.389112 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0914a08b-df47-4f49-aa7d-a8125a2b2c9b admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:35:12.389192 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-ed2a47ba-1526-4a00-84a8-20171fd99138 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:35:12.398181 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1270/2532] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:35:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 11:35:12.402040 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0914a08b-df47-4f49-aa7d-a8125a2b2c9b admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:35:12.413661 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0914a08b-df47-4f49-aa7d-a8125a2b2c9b admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:35:12.418799 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47c25e3-1cfa-4a91-b322-9bd26e41471d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:35:12.420519 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47c25e3-1cfa-4a91-b322-9bd26e41471d admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:35:12.420519 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47c25e3-1cfa-4a91-b322-9bd26e41471d admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:35:12.420519 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47c25e3-1cfa-4a91-b322-9bd26e41471d admin admin] PATH_INFO: `/v3/projects/57fa6350a96a4bd0a64687326785412e/users/298c0093921740d6a7736bfadb42b831/roles/e95c2ce63e114cc19a9dda6ad21e794f` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:35:12.425704 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:35:12.438470 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47c25e3-1cfa-4a91-b322-9bd26e41471d admin admin] RBAC: Authorizing `identity:create_grant(project_id=57fa6350a96a4bd0a64687326785412e, user_id=298c0093921740d6a7736bfadb42b831, role_id=e95c2ce63e114cc19a9dda6ad21e794f)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:35:12.441330 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47c25e3-1cfa-4a91-b322-9bd26e41471d admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:35:12.446205 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1263/2533] 10.222.0.44 () {58 vars in 1018 bytes} [Wed Jun 29 11:35:12 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 11:35:12.458501 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2a47ba-1526-4a00-84a8-20171fd99138 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:35:12.462469 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-ed2a47ba-1526-4a00-84a8-20171fd99138 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:35:12.470834 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2a47ba-1526-4a00-84a8-20171fd99138 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:35:12.474878 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb9c648-1be3-47cc-a4cc-13e3a3a3a2e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f4bb9f989e4024a5da28dced646b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:35:12.475521 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb9c648-1be3-47cc-a4cc-13e3a3a3a2e7 service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:35:12.475802 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb9c648-1be3-47cc-a4cc-13e3a3a3a2e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:35:12.476137 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb9c648-1be3-47cc-a4cc-13e3a3a3a2e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:35:12.483685 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb9c648-1be3-47cc-a4cc-13e3a3a3a2e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:35:12.486309 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb9c648-1be3-47cc-a4cc-13e3a3a3a2e7 service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:35:12.492901 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1264/2534] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:35:12 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 11:35:12.541044 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1271/2535] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jun 29 11:35:12 2022] PUT /identity/v3/projects/57fa6350a96a4bd0a64687326785412e/users/298c0093921740d6a7736bfadb42b831/roles/e95c2ce63e114cc19a9dda6ad21e794f => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 11:35:12.544946 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb9c648-1be3-47cc-a4cc-13e3a3a3a2e7 service neutron] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:35:12.556317 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb9c648-1be3-47cc-a4cc-13e3a3a3a2e7 service neutron] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:35:12.581165 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e554665-3631-4cdd-b325-92b3761c5d9d None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:35:12.581455 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e554665-3631-4cdd-b325-92b3761c5d9d None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:35:12.581736 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e554665-3631-4cdd-b325-92b3761c5d9d None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:35:12.583116 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1272/2536] 10.222.0.44 () {54 vars in 921 bytes} [Wed Jun 29 11:35:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 29 11:35:12.589130 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94c0470-24d7-4224-89f4-a0d20931450c None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:35:12.589487 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94c0470-24d7-4224-89f4-a0d20931450c None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:35:12.589880 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94c0470-24d7-4224-89f4-a0d20931450c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:35:12.608510 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-c94c0470-24d7-4224-89f4-a0d20931450c None None] MFA Rules not processed for user `796aeeff16df4739bddec1f6dee61bfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:35:12.609747 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-c94c0470-24d7-4224-89f4-a0d20931450c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:35:12.611956 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:35:12.618947 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07d9d48-a477-49de-aa32-6a516d194b3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:35:12.619613 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07d9d48-a477-49de-aa32-6a516d194b3e admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:35:12.619934 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07d9d48-a477-49de-aa32-6a516d194b3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:35:12.620214 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07d9d48-a477-49de-aa32-6a516d194b3e admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:35:12.620940 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07d9d48-a477-49de-aa32-6a516d194b3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:35:12.623039 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07d9d48-a477-49de-aa32-6a516d194b3e admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:35:12.632878 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1265/2537] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:35:12 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 11:35:12.636957 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07d9d48-a477-49de-aa32-6a516d194b3e admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:35:12.647600 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07d9d48-a477-49de-aa32-6a516d194b3e admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:35:12.652305 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65507bf4-e3c9-4d34-b533-4d14024d2227 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:35:12.653173 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65507bf4-e3c9-4d34-b533-4d14024d2227 admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:35:12.653637 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65507bf4-e3c9-4d34-b533-4d14024d2227 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:35:12.653921 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65507bf4-e3c9-4d34-b533-4d14024d2227 admin admin] PATH_INFO: `/v3/projects/57fa6350a96a4bd0a64687326785412e/users/298c0093921740d6a7736bfadb42b831/roles/e95c2ce63e114cc19a9dda6ad21e794f` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:35:12.655953 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65507bf4-e3c9-4d34-b533-4d14024d2227 admin admin] RBAC: Authorizing `identity:create_grant(project_id=57fa6350a96a4bd0a64687326785412e, user_id=298c0093921740d6a7736bfadb42b831, role_id=e95c2ce63e114cc19a9dda6ad21e794f)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:35:12.658642 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65507bf4-e3c9-4d34-b533-4d14024d2227 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:35:12.673510 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1266/2538] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jun 29 11:35:12 2022] PUT /identity/v3/projects/57fa6350a96a4bd0a64687326785412e/users/298c0093921740d6a7736bfadb42b831/roles/e95c2ce63e114cc19a9dda6ad21e794f => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 11:35:12.683391 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51bfb44-c7d1-43c8-a75c-2d6dec77477f None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:35:12.683684 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51bfb44-c7d1-43c8-a75c-2d6dec77477f None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:35:12.683992 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51bfb44-c7d1-43c8-a75c-2d6dec77477f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:35:12.687373 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1273/2539] 10.222.0.44 () {58 vars in 1018 bytes} [Wed Jun 29 11:35:12 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 11:35:12.697788 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94c0470-24d7-4224-89f4-a0d20931450c None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:35:12.712440 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-e51bfb44-c7d1-43c8-a75c-2d6dec77477f None None] MFA Rules not processed for user `298c0093921740d6a7736bfadb42b831`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:35:12.713655 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-e51bfb44-c7d1-43c8-a75c-2d6dec77477f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:35:12.759675 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94c0470-24d7-4224-89f4-a0d20931450c None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:35:12.786074 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1267/2540] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:35:12 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 11:35:12.797273 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51bfb44-c7d1-43c8-a75c-2d6dec77477f None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:35:12.798690 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-e51bfb44-c7d1-43c8-a75c-2d6dec77477f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:35:12.808940 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf35c1b6-035e-43d6-8918-0217f3d9b680 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f4bb9f989e4024a5da28dced646b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:35:12.810116 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf35c1b6-035e-43d6-8918-0217f3d9b680 service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:35:12.810408 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf35c1b6-035e-43d6-8918-0217f3d9b680 service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:35:12.810711 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf35c1b6-035e-43d6-8918-0217f3d9b680 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:35:12.855727 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:35:12.858355 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51bfb44-c7d1-43c8-a75c-2d6dec77477f None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:35:12.862176 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5bc0da-61e6-4a45-a0c3-3660d5502a86 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff7931b75a924394b1af733febce0834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:35:12.862806 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5bc0da-61e6-4a45-a0c3-3660d5502a86 service cinder] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:35:12.863087 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5bc0da-61e6-4a45-a0c3-3660d5502a86 service cinder] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:35:12.863368 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5bc0da-61e6-4a45-a0c3-3660d5502a86 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:35:12.873150 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5bc0da-61e6-4a45-a0c3-3660d5502a86 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:35:12.874578 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5bc0da-61e6-4a45-a0c3-3660d5502a86 service cinder] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:35:12.879489 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf35c1b6-035e-43d6-8918-0217f3d9b680 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:35:12.880932 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf35c1b6-035e-43d6-8918-0217f3d9b680 service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:35:12.884072 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1268/2541] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:35:12 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 11:35:12.895349 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1274/2542] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:35:12 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:35:13.078051 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5bc0da-61e6-4a45-a0c3-3660d5502a86 service cinder] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:35:13.088025 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5bc0da-61e6-4a45-a0c3-3660d5502a86 service cinder] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:35:13.092165 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b18b45b-0ca5-4134-9d49-c393f27e5a30 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff7931b75a924394b1af733febce0834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:35:13.092804 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b18b45b-0ca5-4134-9d49-c393f27e5a30 service cinder] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:35:13.093111 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b18b45b-0ca5-4134-9d49-c393f27e5a30 service cinder] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:35:13.093389 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b18b45b-0ca5-4134-9d49-c393f27e5a30 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:35:13.150758 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b18b45b-0ca5-4134-9d49-c393f27e5a30 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:35:13.152143 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b18b45b-0ca5-4134-9d49-c393f27e5a30 service cinder] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:35:13.161117 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1269/2543] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:35:13 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:35:15.325930 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31ce479-1638-4dbe-b175-885c0125a319 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:35:15.326443 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31ce479-1638-4dbe-b175-885c0125a319 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:35:15.326539 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31ce479-1638-4dbe-b175-885c0125a319 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:35:15.349111 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-f31ce479-1638-4dbe-b175-885c0125a319 None None] MFA Rules not processed for user `55487c2107a44c15b656b4865ee0d4fa`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:35:15.350554 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-f31ce479-1638-4dbe-b175-885c0125a319 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:35:15.411825 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1275/2544] 10.222.0.44 () {58 vars in 1009 bytes} [Wed Jun 29 11:35:15 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:35:15.424069 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b18b45b-0ca5-4134-9d49-c393f27e5a30 service cinder] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:35:15.425656 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-6b18b45b-0ca5-4134-9d49-c393f27e5a30 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:35:15.436293 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:35:15.494825 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b18b45b-0ca5-4134-9d49-c393f27e5a30 service cinder] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:35:15.500898 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2894524d-ceaf-4286-84e9-daee9b2aa79d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3b38493ba904759badc800e396befda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:35:15.503517 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2894524d-ceaf-4286-84e9-daee9b2aa79d service swift] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:35:15.503517 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2894524d-ceaf-4286-84e9-daee9b2aa79d service swift] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:35:15.503517 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2894524d-ceaf-4286-84e9-daee9b2aa79d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:35:15.509975 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2894524d-ceaf-4286-84e9-daee9b2aa79d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:35:15.511586 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2894524d-ceaf-4286-84e9-daee9b2aa79d service swift] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:35:15.519779 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1270/2545] 10.222.0.44 () {60 vars in 1384 bytes} [Wed Jun 29 11:35:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 29 11:35:16.604764 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31ce479-1638-4dbe-b175-885c0125a319 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:35:16.674843 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31ce479-1638-4dbe-b175-885c0125a319 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:35:16.679403 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abafd159-917d-4b70-a67b-11e02dbfb3e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:35:16.680308 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abafd159-917d-4b70-a67b-11e02dbfb3e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:35:16.680588 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abafd159-917d-4b70-a67b-11e02dbfb3e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:35:16.680863 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abafd159-917d-4b70-a67b-11e02dbfb3e8 admin admin] PATH_INFO: `/v3/users/5fce995667af44c890cbb5d4d52d67cb` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:35:16.682051 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abafd159-917d-4b70-a67b-11e02dbfb3e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5fce995667af44c890cbb5d4d52d67cb)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:35:16.685128 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abafd159-917d-4b70-a67b-11e02dbfb3e8 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:35:16.908048 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-abafd159-917d-4b70-a67b-11e02dbfb3e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5fce995667af44c890cbb5d4d52d67cb'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:35:16.908849 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-abafd159-917d-4b70-a67b-11e02dbfb3e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5fce995667af44c890cbb5d4d52d67cb'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:35:17.003362 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-abafd159-917d-4b70-a67b-11e02dbfb3e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5fce995667af44c890cbb5d4d52d67cb'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:35:17.010995 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-abafd159-917d-4b70-a67b-11e02dbfb3e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5fce995667af44c890cbb5d4d52d67cb'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:35:17.011654 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-abafd159-917d-4b70-a67b-11e02dbfb3e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5fce995667af44c890cbb5d4d52d67cb'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:35:17.024960 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:35:17.030205 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1276/2546] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jun 29 11:35:16 2022] DELETE /identity/v3/users/5fce995667af44c890cbb5d4d52d67cb => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:35:17.035123 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a684f80d-b6da-40f6-a918-d0a79b2859d2 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:35:17.035642 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a684f80d-b6da-40f6-a918-d0a79b2859d2 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:35:17.035743 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a684f80d-b6da-40f6-a918-d0a79b2859d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:35:17.058340 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-a684f80d-b6da-40f6-a918-d0a79b2859d2 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:35:17.061272 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-a684f80d-b6da-40f6-a918-d0a79b2859d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:35:17.127300 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1271/2547] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:35:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:35:17.137247 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abafd159-917d-4b70-a67b-11e02dbfb3e8 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:35:17.138780 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-abafd159-917d-4b70-a67b-11e02dbfb3e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:35:17.199594 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abafd159-917d-4b70-a67b-11e02dbfb3e8 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:35:17.203547 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bf8b41-842d-4678-993b-0f04b2c7d5bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f4bb9f989e4024a5da28dced646b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:35:17.204238 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bf8b41-842d-4678-993b-0f04b2c7d5bb service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:35:17.204520 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bf8b41-842d-4678-993b-0f04b2c7d5bb service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:35:17.204792 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bf8b41-842d-4678-993b-0f04b2c7d5bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:35:17.215006 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bf8b41-842d-4678-993b-0f04b2c7d5bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:35:17.217039 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bf8b41-842d-4678-993b-0f04b2c7d5bb service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:35:17.230612 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1277/2548] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:35:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 11:35:17.656841 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a684f80d-b6da-40f6-a918-d0a79b2859d2 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:35:17.658400 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-a684f80d-b6da-40f6-a918-d0a79b2859d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:35:17.678561 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a684f80d-b6da-40f6-a918-d0a79b2859d2 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:35:17.691633 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569603c3-d7e0-4de6-b63a-1a69033a9e36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:35:17.692611 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569603c3-d7e0-4de6-b63a-1a69033a9e36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:35:17.692906 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569603c3-d7e0-4de6-b63a-1a69033a9e36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:35:17.693826 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569603c3-d7e0-4de6-b63a-1a69033a9e36 admin admin] PATH_INFO: `/v3/projects/5de11f6ce85a49ed91cc17930759e641` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:35:17.695201 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569603c3-d7e0-4de6-b63a-1a69033a9e36 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5de11f6ce85a49ed91cc17930759e641)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:35:17.697660 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569603c3-d7e0-4de6-b63a-1a69033a9e36 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:35:17.806264 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-569603c3-d7e0-4de6-b63a-1a69033a9e36 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5de11f6ce85a49ed91cc17930759e641', 'request_id': 'req-569603c3-d7e0-4de6-b63a-1a69033a9e36'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:35:17.889096 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-569603c3-d7e0-4de6-b63a-1a69033a9e36 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5de11f6ce85a49ed91cc17930759e641', 'request_id': 'req-569603c3-d7e0-4de6-b63a-1a69033a9e36'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:35:17.895900 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-569603c3-d7e0-4de6-b63a-1a69033a9e36 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5de11f6ce85a49ed91cc17930759e641', 'request_id': 'req-569603c3-d7e0-4de6-b63a-1a69033a9e36'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:35:17.906721 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-569603c3-d7e0-4de6-b63a-1a69033a9e36 admin admin] The token cache is being invalidate because project 5de11f6ce85a49ed91cc17930759e641 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 11:35:17.907007 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-569603c3-d7e0-4de6-b63a-1a69033a9e36 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:35:17.907618 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-569603c3-d7e0-4de6-b63a-1a69033a9e36 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:35:17.909191 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1272/2549] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jun 29 11:35:17 2022] DELETE /identity/v3/projects/5de11f6ce85a49ed91cc17930759e641 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 11:35:17.913452 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bf8b41-842d-4678-993b-0f04b2c7d5bb service neutron] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:35:17.924060 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bf8b41-842d-4678-993b-0f04b2c7d5bb service neutron] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:35:17.988736 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82dd092-c6e6-4407-91a5-d676333b20d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:35:17.989541 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82dd092-c6e6-4407-91a5-d676333b20d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:35:17.989826 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82dd092-c6e6-4407-91a5-d676333b20d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:35:17.990169 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82dd092-c6e6-4407-91a5-d676333b20d5 admin admin] PATH_INFO: `/v3/users/298c0093921740d6a7736bfadb42b831` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:35:17.991353 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82dd092-c6e6-4407-91a5-d676333b20d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=298c0093921740d6a7736bfadb42b831)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:35:17.993312 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82dd092-c6e6-4407-91a5-d676333b20d5 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:35:18.173817 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-b82dd092-c6e6-4407-91a5-d676333b20d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '298c0093921740d6a7736bfadb42b831'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:35:18.174536 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-b82dd092-c6e6-4407-91a5-d676333b20d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '298c0093921740d6a7736bfadb42b831'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:35:18.237653 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-b82dd092-c6e6-4407-91a5-d676333b20d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '298c0093921740d6a7736bfadb42b831'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:35:18.245923 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-b82dd092-c6e6-4407-91a5-d676333b20d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '298c0093921740d6a7736bfadb42b831'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:35:18.246616 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-b82dd092-c6e6-4407-91a5-d676333b20d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '298c0093921740d6a7736bfadb42b831'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:35:18.266958 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1278/2550] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jun 29 11:35:17 2022] DELETE /identity/v3/users/298c0093921740d6a7736bfadb42b831 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:35:18.679723 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569603c3-d7e0-4de6-b63a-1a69033a9e36 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:35:18.682427 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-569603c3-d7e0-4de6-b63a-1a69033a9e36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:35:18.715511 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569603c3-d7e0-4de6-b63a-1a69033a9e36 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:35:18.732473 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:35:18.775152 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14921583-ba9d-47bf-8a6d-6df4c78da705 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:35:18.776056 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14921583-ba9d-47bf-8a6d-6df4c78da705 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:35:18.776398 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14921583-ba9d-47bf-8a6d-6df4c78da705 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:35:18.776674 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14921583-ba9d-47bf-8a6d-6df4c78da705 admin admin] PATH_INFO: `/v3/projects/57fa6350a96a4bd0a64687326785412e` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:35:18.777921 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14921583-ba9d-47bf-8a6d-6df4c78da705 admin admin] RBAC: Authorizing `identity:delete_project(project_id=57fa6350a96a4bd0a64687326785412e)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:35:18.779956 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14921583-ba9d-47bf-8a6d-6df4c78da705 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:35:18.880799 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-14921583-ba9d-47bf-8a6d-6df4c78da705 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '57fa6350a96a4bd0a64687326785412e', 'request_id': 'req-14921583-ba9d-47bf-8a6d-6df4c78da705'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:35:18.989608 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-14921583-ba9d-47bf-8a6d-6df4c78da705 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '57fa6350a96a4bd0a64687326785412e', 'request_id': 'req-14921583-ba9d-47bf-8a6d-6df4c78da705'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:35:18.998086 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-14921583-ba9d-47bf-8a6d-6df4c78da705 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '57fa6350a96a4bd0a64687326785412e', 'request_id': 'req-14921583-ba9d-47bf-8a6d-6df4c78da705'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:35:19.021960 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-14921583-ba9d-47bf-8a6d-6df4c78da705 admin admin] The token cache is being invalidate because project 57fa6350a96a4bd0a64687326785412e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 11:35:19.022374 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-14921583-ba9d-47bf-8a6d-6df4c78da705 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:35:19.023155 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-14921583-ba9d-47bf-8a6d-6df4c78da705 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:35:19.024908 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1273/2551] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jun 29 11:35:18 2022] DELETE /identity/v3/projects/57fa6350a96a4bd0a64687326785412e => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:35:19.031289 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ebe5d9-3c79-4d42-ad2c-db6892af618a None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:35:19.031790 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ebe5d9-3c79-4d42-ad2c-db6892af618a None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:35:19.031790 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ebe5d9-3c79-4d42-ad2c-db6892af618a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:35:19.049677 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-e1ebe5d9-3c79-4d42-ad2c-db6892af618a None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:35:19.050881 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-e1ebe5d9-3c79-4d42-ad2c-db6892af618a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:35:19.065323 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:35:19.119993 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1279/2552] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:35:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 11:35:19.129519 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0c3947-927d-4c31-8447-f879d156bb5e None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:35:19.129819 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0c3947-927d-4c31-8447-f879d156bb5e None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:35:19.130104 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0c3947-927d-4c31-8447-f879d156bb5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:35:19.148319 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-ce0c3947-927d-4c31-8447-f879d156bb5e None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:35:19.154368 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-ce0c3947-927d-4c31-8447-f879d156bb5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:35:19.158700 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1274/2553] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:35:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:35:19.163220 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ebe5d9-3c79-4d42-ad2c-db6892af618a None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:35:19.174145 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ebe5d9-3c79-4d42-ad2c-db6892af618a None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:35:19.179577 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea8d98a-8fd5-4560-8cab-467b952ef761 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:35:19.180270 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea8d98a-8fd5-4560-8cab-467b952ef761 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:35:19.180570 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea8d98a-8fd5-4560-8cab-467b952ef761 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:35:19.180844 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea8d98a-8fd5-4560-8cab-467b952ef761 admin admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:35:19.181562 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea8d98a-8fd5-4560-8cab-467b952ef761 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:35:19.181869 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea8d98a-8fd5-4560-8cab-467b952ef761 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:35:19.183461 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea8d98a-8fd5-4560-8cab-467b952ef761 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:35:19.196118 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1280/2554] 10.222.0.44 () {58 vars in 1152 bytes} [Wed Jun 29 11:35:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:35:19.200516 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0c3947-927d-4c31-8447-f879d156bb5e None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:35:19.209695 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0c3947-927d-4c31-8447-f879d156bb5e None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:35:19.214407 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfff60d-5860-4735-8163-a43f00a724dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:35:19.215050 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfff60d-5860-4735-8163-a43f00a724dd admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:35:19.215336 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfff60d-5860-4735-8163-a43f00a724dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:35:19.215627 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfff60d-5860-4735-8163-a43f00a724dd admin admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:35:19.216388 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfff60d-5860-4735-8163-a43f00a724dd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:35:19.218300 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfff60d-5860-4735-8163-a43f00a724dd admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:35:19.332317 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1275/2555] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jun 29 11:35:19 2022] POST /identity/v3/projects => generated 387 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:35:19.336651 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea8d98a-8fd5-4560-8cab-467b952ef761 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:35:19.348182 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea8d98a-8fd5-4560-8cab-467b952ef761 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:35:19.411069 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560aec68-8567-4344-bc16-8f403aa17369 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:35:19.411733 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560aec68-8567-4344-bc16-8f403aa17369 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:35:19.412045 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560aec68-8567-4344-bc16-8f403aa17369 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:35:19.412328 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560aec68-8567-4344-bc16-8f403aa17369 admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:35:19.413006 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560aec68-8567-4344-bc16-8f403aa17369 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:35:19.414895 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560aec68-8567-4344-bc16-8f403aa17369 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:35:19.517957 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1281/2556] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jun 29 11:35:19 2022] POST /identity/v3/users => generated 294 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 11:35:19.522001 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfff60d-5860-4735-8163-a43f00a724dd admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:35:19.533955 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfff60d-5860-4735-8163-a43f00a724dd admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:35:19.538706 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e0a44d-ba6b-40d3-acea-a98c6062e845 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:35:19.539427 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e0a44d-ba6b-40d3-acea-a98c6062e845 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:35:19.539713 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e0a44d-ba6b-40d3-acea-a98c6062e845 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:35:19.540015 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e0a44d-ba6b-40d3-acea-a98c6062e845 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:35:19.540760 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e0a44d-ba6b-40d3-acea-a98c6062e845 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:35:19.542396 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e0a44d-ba6b-40d3-acea-a98c6062e845 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:35:19.555109 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1276/2557] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:35:19 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 11:35:19.559599 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560aec68-8567-4344-bc16-8f403aa17369 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:35:19.569953 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560aec68-8567-4344-bc16-8f403aa17369 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:35:19.574594 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865a1a4f-c47c-4dd1-b64f-880e44f19329 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:35:19.575254 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865a1a4f-c47c-4dd1-b64f-880e44f19329 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:35:19.575541 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865a1a4f-c47c-4dd1-b64f-880e44f19329 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:35:19.575893 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865a1a4f-c47c-4dd1-b64f-880e44f19329 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:35:19.576631 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865a1a4f-c47c-4dd1-b64f-880e44f19329 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:35:19.578293 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865a1a4f-c47c-4dd1-b64f-880e44f19329 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:35:19.589302 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1282/2558] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:35:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 11:35:19.593804 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e0a44d-ba6b-40d3-acea-a98c6062e845 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:35:19.604282 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e0a44d-ba6b-40d3-acea-a98c6062e845 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:35:19.610009 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b64265-e6ee-494e-ad80-be5c9e76605f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:35:19.611055 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b64265-e6ee-494e-ad80-be5c9e76605f admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:35:19.611338 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b64265-e6ee-494e-ad80-be5c9e76605f admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:35:19.611610 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b64265-e6ee-494e-ad80-be5c9e76605f admin admin] PATH_INFO: `/v3/projects/7ab4db2a3ea447b986461b7bf1304bfb/users/0fa0b2c21c774ba0b6aa6144e4e91ad8/roles/e1c88eebe911432cab6a07fbeb84653b` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:35:19.630681 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b64265-e6ee-494e-ad80-be5c9e76605f admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ab4db2a3ea447b986461b7bf1304bfb, user_id=0fa0b2c21c774ba0b6aa6144e4e91ad8, role_id=e1c88eebe911432cab6a07fbeb84653b)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:35:19.633425 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b64265-e6ee-494e-ad80-be5c9e76605f admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:35:19.698012 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1277/2559] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jun 29 11:35:19 2022] PUT /identity/v3/projects/7ab4db2a3ea447b986461b7bf1304bfb/users/0fa0b2c21c774ba0b6aa6144e4e91ad8/roles/e1c88eebe911432cab6a07fbeb84653b => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 11:35:19.708748 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e948161-9060-4567-b06e-76613f7485df None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:35:19.709084 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e948161-9060-4567-b06e-76613f7485df None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:35:19.709366 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e948161-9060-4567-b06e-76613f7485df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:35:19.739355 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-1e948161-9060-4567-b06e-76613f7485df None None] MFA Rules not processed for user `0fa0b2c21c774ba0b6aa6144e4e91ad8`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:35:19.741593 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-1e948161-9060-4567-b06e-76613f7485df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:35:19.806910 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1283/2560] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:35:19 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:35:19.811404 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b64265-e6ee-494e-ad80-be5c9e76605f admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:35:19.861172 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:35:19.876501 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b64265-e6ee-494e-ad80-be5c9e76605f admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:35:19.881304 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab567d7-0c37-405e-aab8-73d56777c826 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:35:19.881969 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab567d7-0c37-405e-aab8-73d56777c826 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:35:19.882253 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab567d7-0c37-405e-aab8-73d56777c826 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:35:19.882533 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab567d7-0c37-405e-aab8-73d56777c826 admin admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:35:19.883241 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab567d7-0c37-405e-aab8-73d56777c826 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:35:19.885193 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab567d7-0c37-405e-aab8-73d56777c826 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:35:20.005570 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1278/2561] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jun 29 11:35:19 2022] POST /identity/v3/projects => generated 387 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:35:20.009377 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e948161-9060-4567-b06e-76613f7485df None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:35:20.020337 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e948161-9060-4567-b06e-76613f7485df None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:35:20.033496 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:35:20.095006 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9aa6b7c-436e-4451-8d53-17b10e4234b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:35:20.095693 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9aa6b7c-436e-4451-8d53-17b10e4234b4 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:35:20.096012 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9aa6b7c-436e-4451-8d53-17b10e4234b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:35:20.096290 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9aa6b7c-436e-4451-8d53-17b10e4234b4 admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:35:20.096983 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9aa6b7c-436e-4451-8d53-17b10e4234b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:35:20.098944 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9aa6b7c-436e-4451-8d53-17b10e4234b4 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:35:20.169551 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1284/2562] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jun 29 11:35:20 2022] POST /identity/v3/users => generated 300 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:35:20.173588 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab567d7-0c37-405e-aab8-73d56777c826 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:35:20.186630 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab567d7-0c37-405e-aab8-73d56777c826 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:35:20.191938 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f89fbbe-1ef0-48ac-b246-fdd0fd131e77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:35:20.192691 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f89fbbe-1ef0-48ac-b246-fdd0fd131e77 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:35:20.193051 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f89fbbe-1ef0-48ac-b246-fdd0fd131e77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:35:20.193404 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f89fbbe-1ef0-48ac-b246-fdd0fd131e77 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:35:20.194214 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f89fbbe-1ef0-48ac-b246-fdd0fd131e77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:35:20.195907 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f89fbbe-1ef0-48ac-b246-fdd0fd131e77 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:35:20.215347 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1279/2563] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:35:20 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 11:35:20.219605 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9aa6b7c-436e-4451-8d53-17b10e4234b4 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:35:20.229863 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9aa6b7c-436e-4451-8d53-17b10e4234b4 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:35:20.235117 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31a9ea6-7948-4634-9e77-23df01cefa68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:35:20.236030 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31a9ea6-7948-4634-9e77-23df01cefa68 admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:35:20.236318 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31a9ea6-7948-4634-9e77-23df01cefa68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:35:20.236595 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31a9ea6-7948-4634-9e77-23df01cefa68 admin admin] PATH_INFO: `/v3/projects/f9ecd5f777294711b95f108b8b727054/users/a092428a97cf48aa8476f9bd5221b617/roles/e95c2ce63e114cc19a9dda6ad21e794f` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:35:20.252319 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31a9ea6-7948-4634-9e77-23df01cefa68 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9ecd5f777294711b95f108b8b727054, user_id=a092428a97cf48aa8476f9bd5221b617, role_id=e95c2ce63e114cc19a9dda6ad21e794f)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:35:20.255466 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31a9ea6-7948-4634-9e77-23df01cefa68 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:35:20.344086 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1285/2564] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jun 29 11:35:20 2022] PUT /identity/v3/projects/f9ecd5f777294711b95f108b8b727054/users/a092428a97cf48aa8476f9bd5221b617/roles/e95c2ce63e114cc19a9dda6ad21e794f => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 11:35:20.345602 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f89fbbe-1ef0-48ac-b246-fdd0fd131e77 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:35:20.356626 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f89fbbe-1ef0-48ac-b246-fdd0fd131e77 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:35:20.430334 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd2b60e-fc95-4c73-88c6-9e34854b4c07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:35:20.430998 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd2b60e-fc95-4c73-88c6-9e34854b4c07 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:35:20.431285 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd2b60e-fc95-4c73-88c6-9e34854b4c07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:35:20.431565 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd2b60e-fc95-4c73-88c6-9e34854b4c07 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:35:20.432331 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd2b60e-fc95-4c73-88c6-9e34854b4c07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:35:20.434018 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd2b60e-fc95-4c73-88c6-9e34854b4c07 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:35:20.444220 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1280/2565] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:35:20 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 11:35:20.448358 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31a9ea6-7948-4634-9e77-23df01cefa68 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:35:20.459581 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31a9ea6-7948-4634-9e77-23df01cefa68 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:35:20.465546 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c900ef2-50fa-4491-8388-d241785c0cf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:35:20.466560 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c900ef2-50fa-4491-8388-d241785c0cf0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:35:20.466904 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c900ef2-50fa-4491-8388-d241785c0cf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:35:20.467249 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c900ef2-50fa-4491-8388-d241785c0cf0 admin admin] PATH_INFO: `/v3/projects/f9ecd5f777294711b95f108b8b727054/users/a092428a97cf48aa8476f9bd5221b617/roles/e95c2ce63e114cc19a9dda6ad21e794f` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:35:20.469420 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c900ef2-50fa-4491-8388-d241785c0cf0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9ecd5f777294711b95f108b8b727054, user_id=a092428a97cf48aa8476f9bd5221b617, role_id=e95c2ce63e114cc19a9dda6ad21e794f)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:35:20.472282 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c900ef2-50fa-4491-8388-d241785c0cf0 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:35:20.493376 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1286/2566] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jun 29 11:35:20 2022] PUT /identity/v3/projects/f9ecd5f777294711b95f108b8b727054/users/a092428a97cf48aa8476f9bd5221b617/roles/e95c2ce63e114cc19a9dda6ad21e794f => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:35:20.494697 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754c242e-089b-4b4e-b3c6-8aa6d7318ad6 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:35:20.495020 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754c242e-089b-4b4e-b3c6-8aa6d7318ad6 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:35:20.495276 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754c242e-089b-4b4e-b3c6-8aa6d7318ad6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:35:20.522169 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-754c242e-089b-4b4e-b3c6-8aa6d7318ad6 None None] MFA Rules not processed for user `a092428a97cf48aa8476f9bd5221b617`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:35:20.523397 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-754c242e-089b-4b4e-b3c6-8aa6d7318ad6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:35:20.596847 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1281/2567] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:35:20 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:35:20.608411 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c900ef2-50fa-4491-8388-d241785c0cf0 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:35:20.610099 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-9c900ef2-50fa-4491-8388-d241785c0cf0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:35:20.676814 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c900ef2-50fa-4491-8388-d241785c0cf0 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:35:20.683024 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d730685-7a83-4393-8fa3-5ff2d6e7045c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff7931b75a924394b1af733febce0834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:35:20.683696 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d730685-7a83-4393-8fa3-5ff2d6e7045c service cinder] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:35:20.684010 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d730685-7a83-4393-8fa3-5ff2d6e7045c service cinder] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:35:20.684393 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d730685-7a83-4393-8fa3-5ff2d6e7045c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:35:20.696309 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d730685-7a83-4393-8fa3-5ff2d6e7045c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:35:20.697784 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d730685-7a83-4393-8fa3-5ff2d6e7045c service cinder] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:35:20.708366 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1287/2568] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:35:20 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:35:20.839230 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754c242e-089b-4b4e-b3c6-8aa6d7318ad6 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:35:20.888283 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:35:20.914860 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754c242e-089b-4b4e-b3c6-8aa6d7318ad6 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:35:20.926054 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57152cdd-911f-4931-a383-5245a7e8731c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:35:20.926991 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57152cdd-911f-4931-a383-5245a7e8731c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:35:20.927226 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57152cdd-911f-4931-a383-5245a7e8731c admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:35:20.927501 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57152cdd-911f-4931-a383-5245a7e8731c admin admin] PATH_INFO: `/v3/users/0fa0b2c21c774ba0b6aa6144e4e91ad8` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:35:20.928800 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57152cdd-911f-4931-a383-5245a7e8731c admin admin] RBAC: Authorizing `identity:delete_user(user_id=0fa0b2c21c774ba0b6aa6144e4e91ad8)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:35:20.930888 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57152cdd-911f-4931-a383-5245a7e8731c admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:35:21.101814 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-57152cdd-911f-4931-a383-5245a7e8731c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0fa0b2c21c774ba0b6aa6144e4e91ad8'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:35:21.163270 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-57152cdd-911f-4931-a383-5245a7e8731c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0fa0b2c21c774ba0b6aa6144e4e91ad8'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:35:21.163921 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-57152cdd-911f-4931-a383-5245a7e8731c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0fa0b2c21c774ba0b6aa6144e4e91ad8'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:35:21.177428 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-57152cdd-911f-4931-a383-5245a7e8731c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0fa0b2c21c774ba0b6aa6144e4e91ad8'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:35:21.178076 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-57152cdd-911f-4931-a383-5245a7e8731c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0fa0b2c21c774ba0b6aa6144e4e91ad8'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:35:21.190123 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1282/2569] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jun 29 11:35:20 2022] DELETE /identity/v3/users/0fa0b2c21c774ba0b6aa6144e4e91ad8 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:35:21.194647 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf03ad3-fe93-4ba5-b3be-f4c31238c7df None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:35:21.195100 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf03ad3-fe93-4ba5-b3be-f4c31238c7df None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:35:21.195240 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf03ad3-fe93-4ba5-b3be-f4c31238c7df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:35:21.213496 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-bdf03ad3-fe93-4ba5-b3be-f4c31238c7df None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:35:21.214705 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-bdf03ad3-fe93-4ba5-b3be-f4c31238c7df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:35:21.277944 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1288/2570] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:35:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 11:35:21.288958 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57152cdd-911f-4931-a383-5245a7e8731c admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:35:21.290387 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-57152cdd-911f-4931-a383-5245a7e8731c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:35:21.357447 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57152cdd-911f-4931-a383-5245a7e8731c admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:35:21.364630 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249a7ef4-987c-450a-ae51-b11e06a4291e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f4bb9f989e4024a5da28dced646b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:35:21.365988 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249a7ef4-987c-450a-ae51-b11e06a4291e service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:35:21.366272 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249a7ef4-987c-450a-ae51-b11e06a4291e service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:35:21.366553 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249a7ef4-987c-450a-ae51-b11e06a4291e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:35:21.382848 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249a7ef4-987c-450a-ae51-b11e06a4291e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:35:21.384409 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249a7ef4-987c-450a-ae51-b11e06a4291e service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:35:21.395098 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1283/2571] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:35:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 11:35:21.785185 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf03ad3-fe93-4ba5-b3be-f4c31238c7df None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:35:21.786739 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-bdf03ad3-fe93-4ba5-b3be-f4c31238c7df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:35:21.811213 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf03ad3-fe93-4ba5-b3be-f4c31238c7df None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:35:21.815859 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9b7225-943b-414e-9934-4362e7c9503f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:35:21.816760 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9b7225-943b-414e-9934-4362e7c9503f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:35:21.817045 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9b7225-943b-414e-9934-4362e7c9503f admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:35:21.817321 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9b7225-943b-414e-9934-4362e7c9503f admin admin] PATH_INFO: `/v3/projects/7ab4db2a3ea447b986461b7bf1304bfb` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:35:21.818506 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9b7225-943b-414e-9934-4362e7c9503f admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ab4db2a3ea447b986461b7bf1304bfb)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:35:21.820476 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9b7225-943b-414e-9934-4362e7c9503f admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:35:21.827716 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:35:21.954189 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-6a9b7225-943b-414e-9934-4362e7c9503f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ab4db2a3ea447b986461b7bf1304bfb', 'request_id': 'req-6a9b7225-943b-414e-9934-4362e7c9503f'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:35:21.960532 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-6a9b7225-943b-414e-9934-4362e7c9503f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ab4db2a3ea447b986461b7bf1304bfb', 'request_id': 'req-6a9b7225-943b-414e-9934-4362e7c9503f'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:35:21.966008 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-6a9b7225-943b-414e-9934-4362e7c9503f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ab4db2a3ea447b986461b7bf1304bfb', 'request_id': 'req-6a9b7225-943b-414e-9934-4362e7c9503f'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:35:22.072381 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-6a9b7225-943b-414e-9934-4362e7c9503f admin admin] The token cache is being invalidate because project 7ab4db2a3ea447b986461b7bf1304bfb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 11:35:22.072819 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-6a9b7225-943b-414e-9934-4362e7c9503f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:35:22.073375 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-6a9b7225-943b-414e-9934-4362e7c9503f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:35:22.075226 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1289/2572] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jun 29 11:35:21 2022] DELETE /identity/v3/projects/7ab4db2a3ea447b986461b7bf1304bfb => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 11:35:22.079303 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249a7ef4-987c-450a-ae51-b11e06a4291e service neutron] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:35:22.089247 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249a7ef4-987c-450a-ae51-b11e06a4291e service neutron] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:35:22.152076 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a45fc4c-6053-4137-a333-3deb2aed563f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:35:22.153120 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a45fc4c-6053-4137-a333-3deb2aed563f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:35:22.153440 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a45fc4c-6053-4137-a333-3deb2aed563f admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:35:22.153714 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a45fc4c-6053-4137-a333-3deb2aed563f admin admin] PATH_INFO: `/v3/users/a092428a97cf48aa8476f9bd5221b617` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:35:22.154916 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a45fc4c-6053-4137-a333-3deb2aed563f admin admin] RBAC: Authorizing `identity:delete_user(user_id=a092428a97cf48aa8476f9bd5221b617)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:35:22.157038 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a45fc4c-6053-4137-a333-3deb2aed563f admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:35:22.162117 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:35:22.358144 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-4a45fc4c-6053-4137-a333-3deb2aed563f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a092428a97cf48aa8476f9bd5221b617'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:35:22.429130 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-4a45fc4c-6053-4137-a333-3deb2aed563f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a092428a97cf48aa8476f9bd5221b617'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:35:22.430097 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-4a45fc4c-6053-4137-a333-3deb2aed563f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a092428a97cf48aa8476f9bd5221b617'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:35:22.445686 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-4a45fc4c-6053-4137-a333-3deb2aed563f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a092428a97cf48aa8476f9bd5221b617'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:35:22.446519 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-4a45fc4c-6053-4137-a333-3deb2aed563f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a092428a97cf48aa8476f9bd5221b617'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:35:22.462193 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1284/2573] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jun 29 11:35:22 2022] DELETE /identity/v3/users/a092428a97cf48aa8476f9bd5221b617 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 11:35:22.881313 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9b7225-943b-414e-9934-4362e7c9503f admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:35:22.881839 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-6a9b7225-943b-414e-9934-4362e7c9503f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:35:22.895072 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9b7225-943b-414e-9934-4362e7c9503f admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:35:22.975675 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0396360-9a52-4121-8b62-71aa491ad61d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:35:22.976679 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0396360-9a52-4121-8b62-71aa491ad61d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:35:22.977038 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0396360-9a52-4121-8b62-71aa491ad61d admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:35:22.977419 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0396360-9a52-4121-8b62-71aa491ad61d admin admin] PATH_INFO: `/v3/projects/f9ecd5f777294711b95f108b8b727054` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:35:22.978867 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0396360-9a52-4121-8b62-71aa491ad61d admin admin] RBAC: Authorizing `identity:delete_project(project_id=f9ecd5f777294711b95f108b8b727054)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:35:22.980897 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0396360-9a52-4121-8b62-71aa491ad61d admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:35:23.096078 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-f0396360-9a52-4121-8b62-71aa491ad61d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f9ecd5f777294711b95f108b8b727054', 'request_id': 'req-f0396360-9a52-4121-8b62-71aa491ad61d'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:35:23.111365 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-f0396360-9a52-4121-8b62-71aa491ad61d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f9ecd5f777294711b95f108b8b727054', 'request_id': 'req-f0396360-9a52-4121-8b62-71aa491ad61d'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:35:23.119410 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-f0396360-9a52-4121-8b62-71aa491ad61d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f9ecd5f777294711b95f108b8b727054', 'request_id': 'req-f0396360-9a52-4121-8b62-71aa491ad61d'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:35:23.218644 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-f0396360-9a52-4121-8b62-71aa491ad61d admin admin] The token cache is being invalidate because project f9ecd5f777294711b95f108b8b727054 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 11:35:23.219083 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-f0396360-9a52-4121-8b62-71aa491ad61d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:35:23.219819 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-f0396360-9a52-4121-8b62-71aa491ad61d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:35:23.225303 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1290/2574] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jun 29 11:35:22 2022] DELETE /identity/v3/projects/f9ecd5f777294711b95f108b8b727054 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:35:23.229352 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a65cb9-3a25-4d9b-9f42-bb6db199ee21 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:35:23.229352 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a65cb9-3a25-4d9b-9f42-bb6db199ee21 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:35:23.229352 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a65cb9-3a25-4d9b-9f42-bb6db199ee21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:35:23.251390 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-b7a65cb9-3a25-4d9b-9f42-bb6db199ee21 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:35:23.252647 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-b7a65cb9-3a25-4d9b-9f42-bb6db199ee21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:35:23.320604 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1285/2575] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:35:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:35:23.324615 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae3419f-f68b-4815-977d-c7c4ace392cc None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:35:23.324910 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae3419f-f68b-4815-977d-c7c4ace392cc None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:35:23.325222 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae3419f-f68b-4815-977d-c7c4ace392cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:35:23.343958 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-1ae3419f-f68b-4815-977d-c7c4ace392cc None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:35:23.345179 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-1ae3419f-f68b-4815-977d-c7c4ace392cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:35:23.353169 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1291/2576] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:35:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:35:23.358452 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a65cb9-3a25-4d9b-9f42-bb6db199ee21 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:35:23.368901 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a65cb9-3a25-4d9b-9f42-bb6db199ee21 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:35:23.373760 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb11bfb4-9f3b-4d5e-8eb6-784a21b1e019 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:35:23.374429 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb11bfb4-9f3b-4d5e-8eb6-784a21b1e019 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:35:23.374713 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb11bfb4-9f3b-4d5e-8eb6-784a21b1e019 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:35:23.375001 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb11bfb4-9f3b-4d5e-8eb6-784a21b1e019 admin admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:35:23.375731 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb11bfb4-9f3b-4d5e-8eb6-784a21b1e019 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:35:23.376091 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb11bfb4-9f3b-4d5e-8eb6-784a21b1e019 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:35:23.377934 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb11bfb4-9f3b-4d5e-8eb6-784a21b1e019 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:35:23.395870 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1286/2577] 10.222.0.44 () {58 vars in 1152 bytes} [Wed Jun 29 11:35:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:35:23.397363 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae3419f-f68b-4815-977d-c7c4ace392cc None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:35:23.409393 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae3419f-f68b-4815-977d-c7c4ace392cc None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:35:23.414206 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c673077-55bd-4405-9f9a-6fb41bbf81e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:35:23.414872 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c673077-55bd-4405-9f9a-6fb41bbf81e7 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:35:23.415158 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c673077-55bd-4405-9f9a-6fb41bbf81e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:35:23.415434 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c673077-55bd-4405-9f9a-6fb41bbf81e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:35:23.416185 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c673077-55bd-4405-9f9a-6fb41bbf81e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:35:23.418118 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c673077-55bd-4405-9f9a-6fb41bbf81e7 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:35:23.435012 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:35:23.505349 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1292/2578] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jun 29 11:35:23 2022] POST /identity/v3/projects => generated 381 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:35:23.509500 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb11bfb4-9f3b-4d5e-8eb6-784a21b1e019 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:35:23.520464 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb11bfb4-9f3b-4d5e-8eb6-784a21b1e019 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:35:23.596076 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdea910-a251-4692-a4e6-9bf22181976a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:35:23.596750 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdea910-a251-4692-a4e6-9bf22181976a admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:35:23.597032 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdea910-a251-4692-a4e6-9bf22181976a admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:35:23.597307 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdea910-a251-4692-a4e6-9bf22181976a admin admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:35:23.598013 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdea910-a251-4692-a4e6-9bf22181976a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:35:23.599989 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdea910-a251-4692-a4e6-9bf22181976a admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:35:23.678711 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:35:23.684341 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1287/2579] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jun 29 11:35:23 2022] POST /identity/v3/users => generated 291 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:35:23.688958 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c673077-55bd-4405-9f9a-6fb41bbf81e7 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:35:23.699657 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c673077-55bd-4405-9f9a-6fb41bbf81e7 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:35:23.704456 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5e66c3-5a55-4366-a349-0cf03a081043 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:35:23.705197 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5e66c3-5a55-4366-a349-0cf03a081043 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:35:23.705574 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5e66c3-5a55-4366-a349-0cf03a081043 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:35:23.705863 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5e66c3-5a55-4366-a349-0cf03a081043 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:35:23.706585 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5e66c3-5a55-4366-a349-0cf03a081043 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:35:23.708245 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5e66c3-5a55-4366-a349-0cf03a081043 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:35:23.721690 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1293/2580] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:35:23 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 11:35:23.733453 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdea910-a251-4692-a4e6-9bf22181976a admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:35:23.757399 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdea910-a251-4692-a4e6-9bf22181976a admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:35:23.760235 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaa5da1-e91e-44c7-88da-5033a0b58c7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:35:23.761107 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaa5da1-e91e-44c7-88da-5033a0b58c7d admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:35:23.761389 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaa5da1-e91e-44c7-88da-5033a0b58c7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:35:23.761665 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaa5da1-e91e-44c7-88da-5033a0b58c7d admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:35:23.762579 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfaa5da1-e91e-44c7-88da-5033a0b58c7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:35:23.764407 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfaa5da1-e91e-44c7-88da-5033a0b58c7d admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:35:23.775753 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1288/2581] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:35:23 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 11:35:23.780137 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5e66c3-5a55-4366-a349-0cf03a081043 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:35:23.790128 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5e66c3-5a55-4366-a349-0cf03a081043 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:35:23.795492 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a063243b-5fd1-4fcc-8949-31eb6c04d725 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:35:23.796447 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a063243b-5fd1-4fcc-8949-31eb6c04d725 admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:35:23.796734 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a063243b-5fd1-4fcc-8949-31eb6c04d725 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:35:23.797014 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a063243b-5fd1-4fcc-8949-31eb6c04d725 admin admin] PATH_INFO: `/v3/projects/8697e06ed0bd436f80395fa480c4e541/users/ce4a2075a7804398977ccf1eb918b72d/roles/e1c88eebe911432cab6a07fbeb84653b` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:35:23.812389 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a063243b-5fd1-4fcc-8949-31eb6c04d725 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8697e06ed0bd436f80395fa480c4e541, user_id=ce4a2075a7804398977ccf1eb918b72d, role_id=e1c88eebe911432cab6a07fbeb84653b)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:35:23.815047 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a063243b-5fd1-4fcc-8949-31eb6c04d725 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:35:23.883935 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1294/2582] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jun 29 11:35:23 2022] PUT /identity/v3/projects/8697e06ed0bd436f80395fa480c4e541/users/ce4a2075a7804398977ccf1eb918b72d/roles/e1c88eebe911432cab6a07fbeb84653b => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 11:35:23.894268 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b24153b-92f9-46ad-9eee-4271eb0a3045 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:35:23.894616 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b24153b-92f9-46ad-9eee-4271eb0a3045 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:35:23.894921 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b24153b-92f9-46ad-9eee-4271eb0a3045 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:35:23.922539 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-8b24153b-92f9-46ad-9eee-4271eb0a3045 None None] MFA Rules not processed for user `ce4a2075a7804398977ccf1eb918b72d`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:35:23.923788 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-8b24153b-92f9-46ad-9eee-4271eb0a3045 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:35:23.998970 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1289/2583] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:35:23 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:35:24.003621 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a063243b-5fd1-4fcc-8949-31eb6c04d725 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:35:24.081777 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a063243b-5fd1-4fcc-8949-31eb6c04d725 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:35:24.086697 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129db4ac-5324-408c-a953-22ba751357c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:35:24.087428 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129db4ac-5324-408c-a953-22ba751357c4 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:35:24.087724 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129db4ac-5324-408c-a953-22ba751357c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:35:24.088033 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129db4ac-5324-408c-a953-22ba751357c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:35:24.088751 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129db4ac-5324-408c-a953-22ba751357c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:35:24.090689 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129db4ac-5324-408c-a953-22ba751357c4 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:35:24.216147 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1295/2584] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jun 29 11:35:24 2022] POST /identity/v3/projects => generated 379 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:35:24.220766 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b24153b-92f9-46ad-9eee-4271eb0a3045 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:35:24.231913 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b24153b-92f9-46ad-9eee-4271eb0a3045 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:35:24.287248 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336578ee-e10d-4142-9f6c-7df7695ef585 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:35:24.287949 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336578ee-e10d-4142-9f6c-7df7695ef585 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:35:24.288235 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336578ee-e10d-4142-9f6c-7df7695ef585 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:35:24.288510 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336578ee-e10d-4142-9f6c-7df7695ef585 admin admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:35:24.289213 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336578ee-e10d-4142-9f6c-7df7695ef585 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:35:24.291252 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336578ee-e10d-4142-9f6c-7df7695ef585 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:35:24.369252 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1290/2585] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jun 29 11:35:24 2022] POST /identity/v3/users => generated 296 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:35:24.373314 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129db4ac-5324-408c-a953-22ba751357c4 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:35:24.385992 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129db4ac-5324-408c-a953-22ba751357c4 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:35:24.391648 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ee1b8f-623b-4319-96b4-144254dc582f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:35:24.392679 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ee1b8f-623b-4319-96b4-144254dc582f admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:35:24.393097 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ee1b8f-623b-4319-96b4-144254dc582f admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:35:24.393503 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ee1b8f-623b-4319-96b4-144254dc582f admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:35:24.394596 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ee1b8f-623b-4319-96b4-144254dc582f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:35:24.397053 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ee1b8f-623b-4319-96b4-144254dc582f admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:35:24.411318 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1296/2586] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:35:24 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 11:35:24.415311 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336578ee-e10d-4142-9f6c-7df7695ef585 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:35:24.427258 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336578ee-e10d-4142-9f6c-7df7695ef585 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:35:24.432040 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8bf4b3-0ef0-47d7-93e1-ece39bf81d46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:35:24.432956 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8bf4b3-0ef0-47d7-93e1-ece39bf81d46 admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:35:24.433218 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8bf4b3-0ef0-47d7-93e1-ece39bf81d46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:35:24.433493 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8bf4b3-0ef0-47d7-93e1-ece39bf81d46 admin admin] PATH_INFO: `/v3/projects/07f0a337400646cca977fdff30a63cbb/users/78a1c33e195247ffb2c9a5c1d9536807/roles/e95c2ce63e114cc19a9dda6ad21e794f` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:35:24.448128 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8bf4b3-0ef0-47d7-93e1-ece39bf81d46 admin admin] RBAC: Authorizing `identity:create_grant(project_id=07f0a337400646cca977fdff30a63cbb, user_id=78a1c33e195247ffb2c9a5c1d9536807, role_id=e95c2ce63e114cc19a9dda6ad21e794f)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:35:24.451295 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8bf4b3-0ef0-47d7-93e1-ece39bf81d46 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:35:24.524207 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1291/2587] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jun 29 11:35:24 2022] PUT /identity/v3/projects/07f0a337400646cca977fdff30a63cbb/users/78a1c33e195247ffb2c9a5c1d9536807/roles/e95c2ce63e114cc19a9dda6ad21e794f => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 11:35:24.528226 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ee1b8f-623b-4319-96b4-144254dc582f admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:35:24.538865 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ee1b8f-623b-4319-96b4-144254dc582f admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:35:24.597299 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8567ac4-c78b-4eca-b783-f28d42751f90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:35:24.597983 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8567ac4-c78b-4eca-b783-f28d42751f90 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:35:24.598290 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8567ac4-c78b-4eca-b783-f28d42751f90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:35:24.598729 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8567ac4-c78b-4eca-b783-f28d42751f90 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:35:24.599482 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8567ac4-c78b-4eca-b783-f28d42751f90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:35:24.601153 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8567ac4-c78b-4eca-b783-f28d42751f90 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:35:24.612749 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1297/2588] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:35:24 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 11:35:24.617013 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8bf4b3-0ef0-47d7-93e1-ece39bf81d46 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:35:24.628975 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8bf4b3-0ef0-47d7-93e1-ece39bf81d46 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:35:24.633641 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e5c1bc-1061-4757-a4cb-f71871a4c667 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:35:24.634532 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e5c1bc-1061-4757-a4cb-f71871a4c667 admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:35:24.634817 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e5c1bc-1061-4757-a4cb-f71871a4c667 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:35:24.635095 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e5c1bc-1061-4757-a4cb-f71871a4c667 admin admin] PATH_INFO: `/v3/projects/07f0a337400646cca977fdff30a63cbb/users/78a1c33e195247ffb2c9a5c1d9536807/roles/e95c2ce63e114cc19a9dda6ad21e794f` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:35:24.637115 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e5c1bc-1061-4757-a4cb-f71871a4c667 admin admin] RBAC: Authorizing `identity:create_grant(project_id=07f0a337400646cca977fdff30a63cbb, user_id=78a1c33e195247ffb2c9a5c1d9536807, role_id=e95c2ce63e114cc19a9dda6ad21e794f)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:35:24.640571 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e5c1bc-1061-4757-a4cb-f71871a4c667 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:35:24.662883 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1292/2589] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jun 29 11:35:24 2022] PUT /identity/v3/projects/07f0a337400646cca977fdff30a63cbb/users/78a1c33e195247ffb2c9a5c1d9536807/roles/e95c2ce63e114cc19a9dda6ad21e794f => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 11:35:24.664823 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0be45bd-9ba2-418f-ba3c-913bfda0c611 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:35:24.665071 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0be45bd-9ba2-418f-ba3c-913bfda0c611 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:35:24.665372 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0be45bd-9ba2-418f-ba3c-913bfda0c611 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:35:24.693768 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-a0be45bd-9ba2-418f-ba3c-913bfda0c611 None None] MFA Rules not processed for user `78a1c33e195247ffb2c9a5c1d9536807`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:35:24.695039 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-a0be45bd-9ba2-418f-ba3c-913bfda0c611 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:35:24.715700 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:35:24.774352 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1298/2590] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:35:24 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:35:24.782324 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e5c1bc-1061-4757-a4cb-f71871a4c667 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:35:24.784116 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-d0e5c1bc-1061-4757-a4cb-f71871a4c667 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:35:24.833977 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:35:24.868773 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e5c1bc-1061-4757-a4cb-f71871a4c667 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:35:24.873166 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4c607a-9dd8-41d6-a607-8b50a6249122 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff7931b75a924394b1af733febce0834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:35:24.873991 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4c607a-9dd8-41d6-a607-8b50a6249122 service cinder] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:35:24.874280 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4c607a-9dd8-41d6-a607-8b50a6249122 service cinder] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:35:24.874557 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4c607a-9dd8-41d6-a607-8b50a6249122 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:35:24.885015 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4c607a-9dd8-41d6-a607-8b50a6249122 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:35:24.886503 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4c607a-9dd8-41d6-a607-8b50a6249122 service cinder] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:35:24.897703 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1293/2591] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:35:24 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:35:25.494441 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0be45bd-9ba2-418f-ba3c-913bfda0c611 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:35:25.556589 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0be45bd-9ba2-418f-ba3c-913bfda0c611 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:35:25.561806 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c903330d-6eb8-4a67-aaff-8c364cf0d5cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:35:25.562671 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c903330d-6eb8-4a67-aaff-8c364cf0d5cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:35:25.562957 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c903330d-6eb8-4a67-aaff-8c364cf0d5cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:35:25.563237 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c903330d-6eb8-4a67-aaff-8c364cf0d5cc admin admin] PATH_INFO: `/v3/users/ce4a2075a7804398977ccf1eb918b72d` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:35:25.564442 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c903330d-6eb8-4a67-aaff-8c364cf0d5cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce4a2075a7804398977ccf1eb918b72d)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:35:25.566358 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c903330d-6eb8-4a67-aaff-8c364cf0d5cc admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:35:25.744524 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-c903330d-6eb8-4a67-aaff-8c364cf0d5cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce4a2075a7804398977ccf1eb918b72d'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:35:25.745449 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-c903330d-6eb8-4a67-aaff-8c364cf0d5cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce4a2075a7804398977ccf1eb918b72d'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:35:25.834726 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-c903330d-6eb8-4a67-aaff-8c364cf0d5cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce4a2075a7804398977ccf1eb918b72d'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:35:25.842254 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-c903330d-6eb8-4a67-aaff-8c364cf0d5cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce4a2075a7804398977ccf1eb918b72d'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:35:25.842886 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-c903330d-6eb8-4a67-aaff-8c364cf0d5cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce4a2075a7804398977ccf1eb918b72d'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:35:25.860438 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1299/2592] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jun 29 11:35:25 2022] DELETE /identity/v3/users/ce4a2075a7804398977ccf1eb918b72d => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:35:25.865009 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fe0865-33dc-47b0-adcd-dc6cb830af5e None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:35:25.865489 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fe0865-33dc-47b0-adcd-dc6cb830af5e None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:35:25.865603 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fe0865-33dc-47b0-adcd-dc6cb830af5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:35:25.882928 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-83fe0865-33dc-47b0-adcd-dc6cb830af5e None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:35:25.884168 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-83fe0865-33dc-47b0-adcd-dc6cb830af5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:35:25.957460 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1294/2593] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:35:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:35:25.967190 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c903330d-6eb8-4a67-aaff-8c364cf0d5cc admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:35:25.968702 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-c903330d-6eb8-4a67-aaff-8c364cf0d5cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:35:26.009058 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:35:26.025686 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c903330d-6eb8-4a67-aaff-8c364cf0d5cc admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:35:26.029338 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7271cdc-d622-4516-a38b-a753bd776383 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f4bb9f989e4024a5da28dced646b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:35:26.029975 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7271cdc-d622-4516-a38b-a753bd776383 service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:35:26.030257 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7271cdc-d622-4516-a38b-a753bd776383 service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:35:26.030531 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7271cdc-d622-4516-a38b-a753bd776383 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:35:26.041160 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7271cdc-d622-4516-a38b-a753bd776383 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:35:26.043735 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7271cdc-d622-4516-a38b-a753bd776383 service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:35:26.055497 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1300/2594] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:35:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:35:26.551148 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fe0865-33dc-47b0-adcd-dc6cb830af5e None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:35:26.552881 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-83fe0865-33dc-47b0-adcd-dc6cb830af5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:35:26.583310 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fe0865-33dc-47b0-adcd-dc6cb830af5e None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:35:26.588152 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43be7a2d-1ef9-405f-a0e9-4f8334d9bb58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:35:26.589047 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43be7a2d-1ef9-405f-a0e9-4f8334d9bb58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:35:26.589463 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43be7a2d-1ef9-405f-a0e9-4f8334d9bb58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:35:26.589746 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43be7a2d-1ef9-405f-a0e9-4f8334d9bb58 admin admin] PATH_INFO: `/v3/projects/8697e06ed0bd436f80395fa480c4e541` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:35:26.590928 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43be7a2d-1ef9-405f-a0e9-4f8334d9bb58 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8697e06ed0bd436f80395fa480c4e541)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:35:26.593455 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43be7a2d-1ef9-405f-a0e9-4f8334d9bb58 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:35:26.697210 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-43be7a2d-1ef9-405f-a0e9-4f8334d9bb58 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8697e06ed0bd436f80395fa480c4e541', 'request_id': 'req-43be7a2d-1ef9-405f-a0e9-4f8334d9bb58'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:35:26.769314 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-43be7a2d-1ef9-405f-a0e9-4f8334d9bb58 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8697e06ed0bd436f80395fa480c4e541', 'request_id': 'req-43be7a2d-1ef9-405f-a0e9-4f8334d9bb58'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:35:26.776977 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-43be7a2d-1ef9-405f-a0e9-4f8334d9bb58 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8697e06ed0bd436f80395fa480c4e541', 'request_id': 'req-43be7a2d-1ef9-405f-a0e9-4f8334d9bb58'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:35:26.788310 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-43be7a2d-1ef9-405f-a0e9-4f8334d9bb58 admin admin] The token cache is being invalidate because project 8697e06ed0bd436f80395fa480c4e541 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 11:35:26.788599 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-43be7a2d-1ef9-405f-a0e9-4f8334d9bb58 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:35:26.789264 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-43be7a2d-1ef9-405f-a0e9-4f8334d9bb58 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:35:26.790863 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1295/2595] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jun 29 11:35:26 2022] DELETE /identity/v3/projects/8697e06ed0bd436f80395fa480c4e541 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:35:26.794937 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7271cdc-d622-4516-a38b-a753bd776383 service neutron] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:35:26.804675 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7271cdc-d622-4516-a38b-a753bd776383 service neutron] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:35:26.866954 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9242b6e4-c99d-4090-9c95-adecb2f03318 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:35:26.867884 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9242b6e4-c99d-4090-9c95-adecb2f03318 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:35:26.868168 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9242b6e4-c99d-4090-9c95-adecb2f03318 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:35:26.868443 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9242b6e4-c99d-4090-9c95-adecb2f03318 admin admin] PATH_INFO: `/v3/users/78a1c33e195247ffb2c9a5c1d9536807` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:35:26.869757 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9242b6e4-c99d-4090-9c95-adecb2f03318 admin admin] RBAC: Authorizing `identity:delete_user(user_id=78a1c33e195247ffb2c9a5c1d9536807)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:35:26.871676 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9242b6e4-c99d-4090-9c95-adecb2f03318 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:35:27.089613 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-9242b6e4-c99d-4090-9c95-adecb2f03318 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '78a1c33e195247ffb2c9a5c1d9536807'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:35:27.090257 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-9242b6e4-c99d-4090-9c95-adecb2f03318 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '78a1c33e195247ffb2c9a5c1d9536807'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:35:27.174596 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-9242b6e4-c99d-4090-9c95-adecb2f03318 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '78a1c33e195247ffb2c9a5c1d9536807'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:35:27.182799 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-9242b6e4-c99d-4090-9c95-adecb2f03318 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '78a1c33e195247ffb2c9a5c1d9536807'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:35:27.183473 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-9242b6e4-c99d-4090-9c95-adecb2f03318 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '78a1c33e195247ffb2c9a5c1d9536807'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:35:27.207202 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1301/2596] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jun 29 11:35:26 2022] DELETE /identity/v3/users/78a1c33e195247ffb2c9a5c1d9536807 => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:35:27.745859 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43be7a2d-1ef9-405f-a0e9-4f8334d9bb58 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:35:27.748127 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-43be7a2d-1ef9-405f-a0e9-4f8334d9bb58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:35:27.766226 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43be7a2d-1ef9-405f-a0e9-4f8334d9bb58 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:35:27.826067 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:35:27.832342 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbb6544-e976-4569-8de5-e237aecba74e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:35:27.833203 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbb6544-e976-4569-8de5-e237aecba74e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:35:27.833483 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbb6544-e976-4569-8de5-e237aecba74e admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:35:27.833757 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbb6544-e976-4569-8de5-e237aecba74e admin admin] PATH_INFO: `/v3/projects/07f0a337400646cca977fdff30a63cbb` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:35:27.834953 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbb6544-e976-4569-8de5-e237aecba74e admin admin] RBAC: Authorizing `identity:delete_project(project_id=07f0a337400646cca977fdff30a63cbb)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:35:27.836937 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbb6544-e976-4569-8de5-e237aecba74e admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:35:27.949122 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-4cbb6544-e976-4569-8de5-e237aecba74e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '07f0a337400646cca977fdff30a63cbb', 'request_id': 'req-4cbb6544-e976-4569-8de5-e237aecba74e'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:35:28.011706 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-4cbb6544-e976-4569-8de5-e237aecba74e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '07f0a337400646cca977fdff30a63cbb', 'request_id': 'req-4cbb6544-e976-4569-8de5-e237aecba74e'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:35:28.022808 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-4cbb6544-e976-4569-8de5-e237aecba74e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '07f0a337400646cca977fdff30a63cbb', 'request_id': 'req-4cbb6544-e976-4569-8de5-e237aecba74e'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:35:28.034352 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-4cbb6544-e976-4569-8de5-e237aecba74e admin admin] The token cache is being invalidate because project 07f0a337400646cca977fdff30a63cbb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 11:35:28.034646 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-4cbb6544-e976-4569-8de5-e237aecba74e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:35:28.035326 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-4cbb6544-e976-4569-8de5-e237aecba74e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:35:28.036927 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1296/2597] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jun 29 11:35:27 2022] DELETE /identity/v3/projects/07f0a337400646cca977fdff30a63cbb => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:35:28.042564 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452992e9-1e53-43c2-afef-d8407e6ccdea None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:35:28.043102 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452992e9-1e53-43c2-afef-d8407e6ccdea None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:35:28.043206 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452992e9-1e53-43c2-afef-d8407e6ccdea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:35:28.065023 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-452992e9-1e53-43c2-afef-d8407e6ccdea None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:35:28.066242 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-452992e9-1e53-43c2-afef-d8407e6ccdea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:35:28.127532 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:35:28.137136 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1302/2598] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:35:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:35:28.145320 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2c65fd-744a-4925-ba05-bab9d02534a8 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:35:28.145616 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2c65fd-744a-4925-ba05-bab9d02534a8 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:35:28.145900 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2c65fd-744a-4925-ba05-bab9d02534a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:35:28.165053 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-5b2c65fd-744a-4925-ba05-bab9d02534a8 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:35:28.167251 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-5b2c65fd-744a-4925-ba05-bab9d02534a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:35:28.174510 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1297/2599] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:35:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:35:28.183593 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452992e9-1e53-43c2-afef-d8407e6ccdea None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:35:28.196035 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452992e9-1e53-43c2-afef-d8407e6ccdea None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:35:28.201942 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bcd635-4b2b-4872-a254-999da0618dd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:35:28.203381 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bcd635-4b2b-4872-a254-999da0618dd6 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:35:28.203918 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bcd635-4b2b-4872-a254-999da0618dd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:35:28.204272 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bcd635-4b2b-4872-a254-999da0618dd6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:35:28.205180 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bcd635-4b2b-4872-a254-999da0618dd6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:35:28.205556 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bcd635-4b2b-4872-a254-999da0618dd6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:35:28.207490 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bcd635-4b2b-4872-a254-999da0618dd6 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:35:28.224672 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1303/2600] 10.222.0.44 () {58 vars in 1152 bytes} [Wed Jun 29 11:35:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:35:28.228926 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2c65fd-744a-4925-ba05-bab9d02534a8 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:35:28.238707 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2c65fd-744a-4925-ba05-bab9d02534a8 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:35:28.243361 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f0fc27-b04f-49ca-93be-1da86fbda377 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:35:28.244063 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f0fc27-b04f-49ca-93be-1da86fbda377 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:35:28.244352 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f0fc27-b04f-49ca-93be-1da86fbda377 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:35:28.244678 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f0fc27-b04f-49ca-93be-1da86fbda377 admin admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:35:28.245387 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f0fc27-b04f-49ca-93be-1da86fbda377 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:35:28.247331 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f0fc27-b04f-49ca-93be-1da86fbda377 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:35:28.363156 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1298/2601] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jun 29 11:35:28 2022] POST /identity/v3/projects => generated 375 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:35:28.367524 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bcd635-4b2b-4872-a254-999da0618dd6 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:35:28.377755 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bcd635-4b2b-4872-a254-999da0618dd6 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:35:28.446291 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f988f43-0639-4b8d-a82b-b236691f77c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:35:28.447001 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f988f43-0639-4b8d-a82b-b236691f77c0 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:35:28.447331 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f988f43-0639-4b8d-a82b-b236691f77c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:35:28.447661 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f988f43-0639-4b8d-a82b-b236691f77c0 admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:35:28.448405 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f988f43-0639-4b8d-a82b-b236691f77c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:35:28.450322 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f988f43-0639-4b8d-a82b-b236691f77c0 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:35:28.533644 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1304/2602] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jun 29 11:35:28 2022] POST /identity/v3/users => generated 288 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:35:28.537765 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f0fc27-b04f-49ca-93be-1da86fbda377 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:35:28.550329 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f0fc27-b04f-49ca-93be-1da86fbda377 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:35:28.556298 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44207b7a-169e-4bb9-91e5-a62a3860f2ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:35:28.557358 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44207b7a-169e-4bb9-91e5-a62a3860f2ff admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:35:28.557820 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44207b7a-169e-4bb9-91e5-a62a3860f2ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:35:28.558101 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44207b7a-169e-4bb9-91e5-a62a3860f2ff admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:35:28.559058 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44207b7a-169e-4bb9-91e5-a62a3860f2ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:35:28.561029 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44207b7a-169e-4bb9-91e5-a62a3860f2ff admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:35:28.577419 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1299/2603] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:35:28 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 11:35:28.578987 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f988f43-0639-4b8d-a82b-b236691f77c0 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:35:28.589806 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f988f43-0639-4b8d-a82b-b236691f77c0 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:35:28.595412 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94ceb17-67ec-4a7b-b54a-77980a058678 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:35:28.596121 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94ceb17-67ec-4a7b-b54a-77980a058678 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:35:28.596406 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94ceb17-67ec-4a7b-b54a-77980a058678 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:35:28.596679 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94ceb17-67ec-4a7b-b54a-77980a058678 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:35:28.597415 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94ceb17-67ec-4a7b-b54a-77980a058678 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:35:28.598983 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94ceb17-67ec-4a7b-b54a-77980a058678 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:35:28.610880 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1305/2604] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:35:28 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 11:35:28.615328 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44207b7a-169e-4bb9-91e5-a62a3860f2ff admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:35:28.625905 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44207b7a-169e-4bb9-91e5-a62a3860f2ff admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:35:28.632711 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797c7072-3f8f-4d95-940b-e126df460a26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:35:28.639885 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797c7072-3f8f-4d95-940b-e126df460a26 admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:35:28.639885 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797c7072-3f8f-4d95-940b-e126df460a26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:35:28.639885 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797c7072-3f8f-4d95-940b-e126df460a26 admin admin] PATH_INFO: `/v3/projects/96ae1f236dd6460788f5d573e780a5d8/users/0121d9b26f344f7c8a8034581a336c30/roles/e1c88eebe911432cab6a07fbeb84653b` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:35:28.651170 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797c7072-3f8f-4d95-940b-e126df460a26 admin admin] RBAC: Authorizing `identity:create_grant(project_id=96ae1f236dd6460788f5d573e780a5d8, user_id=0121d9b26f344f7c8a8034581a336c30, role_id=e1c88eebe911432cab6a07fbeb84653b)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:35:28.654646 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797c7072-3f8f-4d95-940b-e126df460a26 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:35:28.724908 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1300/2605] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jun 29 11:35:28 2022] PUT /identity/v3/projects/96ae1f236dd6460788f5d573e780a5d8/users/0121d9b26f344f7c8a8034581a336c30/roles/e1c88eebe911432cab6a07fbeb84653b => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:35:28.746249 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f9cb2c-6540-4a97-a783-c117305815a3 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:35:28.746541 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f9cb2c-6540-4a97-a783-c117305815a3 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:35:28.746826 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f9cb2c-6540-4a97-a783-c117305815a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:35:28.775427 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-70f9cb2c-6540-4a97-a783-c117305815a3 None None] MFA Rules not processed for user `0121d9b26f344f7c8a8034581a336c30`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:35:28.776725 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-70f9cb2c-6540-4a97-a783-c117305815a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:35:28.845232 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1306/2606] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:35:28 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:35:28.855652 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797c7072-3f8f-4d95-940b-e126df460a26 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:35:28.857811 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-797c7072-3f8f-4d95-940b-e126df460a26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:35:28.923393 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797c7072-3f8f-4d95-940b-e126df460a26 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:35:28.927463 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9face9c3-2dee-4a36-9374-4251807a3282 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede3a17c7063432e8ee66462766584f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:35:28.931757 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9face9c3-2dee-4a36-9374-4251807a3282 service glance] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:35:28.931757 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9face9c3-2dee-4a36-9374-4251807a3282 service glance] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:35:28.931757 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9face9c3-2dee-4a36-9374-4251807a3282 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:35:28.942544 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9face9c3-2dee-4a36-9374-4251807a3282 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:35:28.944284 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9face9c3-2dee-4a36-9374-4251807a3282 service glance] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:35:28.955483 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1301/2607] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:35:28 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:35:38.841027 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=73,keystone:INSERT=3 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:35:38.950272 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=78,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:35:46.663572 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f9cb2c-6540-4a97-a783-c117305815a3 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:35:46.665471 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-70f9cb2c-6540-4a97-a783-c117305815a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:35:46.740400 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f9cb2c-6540-4a97-a783-c117305815a3 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:35:46.745876 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44819c4b-6733-4924-a697-4d87c3c41ca4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '796aeeff16df4739bddec1f6dee61bfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:35:46.746521 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44819c4b-6733-4924-a697-4d87c3c41ca4 service nova] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:35:46.746807 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44819c4b-6733-4924-a697-4d87c3c41ca4 service nova] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:35:46.747089 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44819c4b-6733-4924-a697-4d87c3c41ca4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:35:46.748904 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-44819c4b-6733-4924-a697-4d87c3c41ca4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:35:46.761877 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44819c4b-6733-4924-a697-4d87c3c41ca4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:35:46.763502 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44819c4b-6733-4924-a697-4d87c3c41ca4 service nova] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:35:46.775214 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1307/2608] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:35:46 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:35:47.184429 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9face9c3-2dee-4a36-9374-4251807a3282 service glance] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:35:47.185819 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-9face9c3-2dee-4a36-9374-4251807a3282 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:35:47.245908 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9face9c3-2dee-4a36-9374-4251807a3282 service glance] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:35:47.249453 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4553ef3b-a5a2-4990-b8f0-e03bf28f0063 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f4bb9f989e4024a5da28dced646b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:35:47.250623 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4553ef3b-a5a2-4990-b8f0-e03bf28f0063 service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:35:47.250905 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4553ef3b-a5a2-4990-b8f0-e03bf28f0063 service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:35:47.251177 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4553ef3b-a5a2-4990-b8f0-e03bf28f0063 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:35:47.252976 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-4553ef3b-a5a2-4990-b8f0-e03bf28f0063 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:35:47.262147 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4553ef3b-a5a2-4990-b8f0-e03bf28f0063 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:35:47.263560 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4553ef3b-a5a2-4990-b8f0-e03bf28f0063 service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:35:47.272036 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1302/2609] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:35:47 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:35:49.478263 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b80aca4-8c3b-4d8a-b43d-36aedbc8dccb None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:35:49.478720 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b80aca4-8c3b-4d8a-b43d-36aedbc8dccb None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:35:49.478841 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b80aca4-8c3b-4d8a-b43d-36aedbc8dccb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:35:49.498361 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-2b80aca4-8c3b-4d8a-b43d-36aedbc8dccb None None] MFA Rules not processed for user `55487c2107a44c15b656b4865ee0d4fa`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:35:49.499573 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-2b80aca4-8c3b-4d8a-b43d-36aedbc8dccb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:35:49.553833 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1308/2610] 10.222.0.44 () {58 vars in 967 bytes} [Wed Jun 29 11:35:49 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:35:49.566490 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4553ef3b-a5a2-4990-b8f0-e03bf28f0063 service neutron] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:35:49.567928 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-4553ef3b-a5a2-4990-b8f0-e03bf28f0063 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:35:49.636400 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4553ef3b-a5a2-4990-b8f0-e03bf28f0063 service neutron] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:35:49.641384 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a05c18-1feb-4c5e-99be-05801c591bf9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3b38493ba904759badc800e396befda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:35:49.642023 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a05c18-1feb-4c5e-99be-05801c591bf9 service swift] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:35:49.642307 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a05c18-1feb-4c5e-99be-05801c591bf9 service swift] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:35:49.642586 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a05c18-1feb-4c5e-99be-05801c591bf9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:35:49.650447 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a05c18-1feb-4c5e-99be-05801c591bf9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:35:49.652054 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a05c18-1feb-4c5e-99be-05801c591bf9 service swift] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:35:49.659934 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1303/2611] 10.222.0.44 () {60 vars in 1384 bytes} [Wed Jun 29 11:35:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 29 11:35:57.743760 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b80aca4-8c3b-4d8a-b43d-36aedbc8dccb None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:35:57.807525 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b80aca4-8c3b-4d8a-b43d-36aedbc8dccb None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:35:57.812065 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0790e032-3167-40bd-8fe6-9d364a49aab8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:35:57.812928 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0790e032-3167-40bd-8fe6-9d364a49aab8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:35:57.813212 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0790e032-3167-40bd-8fe6-9d364a49aab8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:35:57.813518 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0790e032-3167-40bd-8fe6-9d364a49aab8 admin admin] PATH_INFO: `/v3/users/0121d9b26f344f7c8a8034581a336c30` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:35:57.814737 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0790e032-3167-40bd-8fe6-9d364a49aab8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0121d9b26f344f7c8a8034581a336c30)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:35:57.817301 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0790e032-3167-40bd-8fe6-9d364a49aab8 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:35:57.985184 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:35:57.997317 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-0790e032-3167-40bd-8fe6-9d364a49aab8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0121d9b26f344f7c8a8034581a336c30'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:35:57.997983 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-0790e032-3167-40bd-8fe6-9d364a49aab8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0121d9b26f344f7c8a8034581a336c30'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:35:58.062564 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-0790e032-3167-40bd-8fe6-9d364a49aab8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0121d9b26f344f7c8a8034581a336c30'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:35:58.070165 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-0790e032-3167-40bd-8fe6-9d364a49aab8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0121d9b26f344f7c8a8034581a336c30'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:35:58.070786 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-0790e032-3167-40bd-8fe6-9d364a49aab8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0121d9b26f344f7c8a8034581a336c30'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:35:58.088043 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1309/2612] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jun 29 11:35:57 2022] DELETE /identity/v3/users/0121d9b26f344f7c8a8034581a336c30 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:35:58.092562 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa492152-5d5c-4500-bd52-5ac55573f970 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:35:58.098781 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa492152-5d5c-4500-bd52-5ac55573f970 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:35:58.098781 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa492152-5d5c-4500-bd52-5ac55573f970 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:35:58.110045 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-fa492152-5d5c-4500-bd52-5ac55573f970 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:35:58.111245 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-fa492152-5d5c-4500-bd52-5ac55573f970 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:35:58.168296 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1304/2613] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:35:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:35:58.179337 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0790e032-3167-40bd-8fe6-9d364a49aab8 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:35:58.180794 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-0790e032-3167-40bd-8fe6-9d364a49aab8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:35:58.241154 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0790e032-3167-40bd-8fe6-9d364a49aab8 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:35:58.244685 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d606e6-2348-45d6-897b-cafb92139d34 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f4bb9f989e4024a5da28dced646b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:35:58.245324 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d606e6-2348-45d6-897b-cafb92139d34 service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:35:58.245605 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d606e6-2348-45d6-897b-cafb92139d34 service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:35:58.245878 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d606e6-2348-45d6-897b-cafb92139d34 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:35:58.254552 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d606e6-2348-45d6-897b-cafb92139d34 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:35:58.255985 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d606e6-2348-45d6-897b-cafb92139d34 service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:35:58.265143 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1310/2614] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:35:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:35:58.655446 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa492152-5d5c-4500-bd52-5ac55573f970 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:35:58.656965 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-fa492152-5d5c-4500-bd52-5ac55573f970 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:35:58.669846 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa492152-5d5c-4500-bd52-5ac55573f970 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:35:58.675076 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b949bd04-19ce-491b-9b70-a313367a1d25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:35:58.676018 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b949bd04-19ce-491b-9b70-a313367a1d25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:35:58.676304 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b949bd04-19ce-491b-9b70-a313367a1d25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:35:58.676587 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b949bd04-19ce-491b-9b70-a313367a1d25 admin admin] PATH_INFO: `/v3/projects/96ae1f236dd6460788f5d573e780a5d8` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:35:58.677785 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b949bd04-19ce-491b-9b70-a313367a1d25 admin admin] RBAC: Authorizing `identity:delete_project(project_id=96ae1f236dd6460788f5d573e780a5d8)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:35:58.679760 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b949bd04-19ce-491b-9b70-a313367a1d25 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:35:58.771203 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:35:58.787417 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-b949bd04-19ce-491b-9b70-a313367a1d25 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '96ae1f236dd6460788f5d573e780a5d8', 'request_id': 'req-b949bd04-19ce-491b-9b70-a313367a1d25'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:35:58.876991 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-b949bd04-19ce-491b-9b70-a313367a1d25 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '96ae1f236dd6460788f5d573e780a5d8', 'request_id': 'req-b949bd04-19ce-491b-9b70-a313367a1d25'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:35:58.883584 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-b949bd04-19ce-491b-9b70-a313367a1d25 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '96ae1f236dd6460788f5d573e780a5d8', 'request_id': 'req-b949bd04-19ce-491b-9b70-a313367a1d25'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:35:58.894707 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-b949bd04-19ce-491b-9b70-a313367a1d25 admin admin] The token cache is being invalidate because project 96ae1f236dd6460788f5d573e780a5d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 11:35:58.895005 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-b949bd04-19ce-491b-9b70-a313367a1d25 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:35:58.895656 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-b949bd04-19ce-491b-9b70-a313367a1d25 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:35:58.897243 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1305/2615] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jun 29 11:35:58 2022] DELETE /identity/v3/projects/96ae1f236dd6460788f5d573e780a5d8 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:35:58.902919 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55a4952-55d1-4fd5-848a-0737a1316de0 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:35:58.904540 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55a4952-55d1-4fd5-848a-0737a1316de0 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:35:58.904540 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55a4952-55d1-4fd5-848a-0737a1316de0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:35:58.927549 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-f55a4952-55d1-4fd5-848a-0737a1316de0 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:35:58.927549 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-f55a4952-55d1-4fd5-848a-0737a1316de0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:35:58.983444 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1311/2616] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:35:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:35:58.991331 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df61910-28ba-414f-9d73-81cad377f8c3 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:35:58.991616 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df61910-28ba-414f-9d73-81cad377f8c3 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:35:58.991924 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df61910-28ba-414f-9d73-81cad377f8c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:35:59.010255 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-5df61910-28ba-414f-9d73-81cad377f8c3 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:35:59.011438 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-5df61910-28ba-414f-9d73-81cad377f8c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:35:59.019710 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1306/2617] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:35:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:35:59.023815 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55a4952-55d1-4fd5-848a-0737a1316de0 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:35:59.033306 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55a4952-55d1-4fd5-848a-0737a1316de0 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:35:59.037817 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8999595-c3aa-4611-8505-79bd59b240e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:35:59.038470 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8999595-c3aa-4611-8505-79bd59b240e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:35:59.038754 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8999595-c3aa-4611-8505-79bd59b240e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:35:59.039031 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8999595-c3aa-4611-8505-79bd59b240e7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:35:59.039743 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8999595-c3aa-4611-8505-79bd59b240e7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:35:59.040086 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8999595-c3aa-4611-8505-79bd59b240e7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:35:59.041675 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8999595-c3aa-4611-8505-79bd59b240e7 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:35:59.052996 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1312/2618] 10.222.0.44 () {58 vars in 1152 bytes} [Wed Jun 29 11:35:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:35:59.057156 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df61910-28ba-414f-9d73-81cad377f8c3 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:35:59.066707 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df61910-28ba-414f-9d73-81cad377f8c3 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:35:59.071255 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ec83b2-e879-467c-bec3-19b148435cca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:35:59.071918 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ec83b2-e879-467c-bec3-19b148435cca admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:35:59.072202 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ec83b2-e879-467c-bec3-19b148435cca admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:35:59.072475 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ec83b2-e879-467c-bec3-19b148435cca admin admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:35:59.073157 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ec83b2-e879-467c-bec3-19b148435cca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:35:59.075041 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ec83b2-e879-467c-bec3-19b148435cca admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:35:59.180933 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1307/2619] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jun 29 11:35:59 2022] POST /identity/v3/projects => generated 371 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 11:35:59.184896 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8999595-c3aa-4611-8505-79bd59b240e7 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:35:59.195803 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8999595-c3aa-4611-8505-79bd59b240e7 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:35:59.234095 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:35:59.256280 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39749110-7ca7-408e-ac77-cce13e4fd645 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:35:59.256968 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39749110-7ca7-408e-ac77-cce13e4fd645 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:35:59.257255 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39749110-7ca7-408e-ac77-cce13e4fd645 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:35:59.257532 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39749110-7ca7-408e-ac77-cce13e4fd645 admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:35:59.258234 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39749110-7ca7-408e-ac77-cce13e4fd645 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:35:59.260811 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39749110-7ca7-408e-ac77-cce13e4fd645 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:35:59.380544 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1313/2620] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jun 29 11:35:59 2022] POST /identity/v3/users => generated 286 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 11:35:59.384602 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ec83b2-e879-467c-bec3-19b148435cca admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:35:59.394873 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ec83b2-e879-467c-bec3-19b148435cca admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:35:59.399362 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b853db27-aa00-426a-abfd-83b874d58329 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:35:59.400051 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b853db27-aa00-426a-abfd-83b874d58329 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:35:59.400340 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b853db27-aa00-426a-abfd-83b874d58329 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:35:59.400619 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b853db27-aa00-426a-abfd-83b874d58329 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:35:59.401378 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b853db27-aa00-426a-abfd-83b874d58329 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:35:59.402968 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b853db27-aa00-426a-abfd-83b874d58329 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:35:59.413278 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1308/2621] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:35:59 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 11:35:59.417273 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39749110-7ca7-408e-ac77-cce13e4fd645 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:35:59.427658 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39749110-7ca7-408e-ac77-cce13e4fd645 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:35:59.432268 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08402d36-d099-4363-8d07-3826651e4d3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:35:59.432910 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08402d36-d099-4363-8d07-3826651e4d3a admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:35:59.433194 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08402d36-d099-4363-8d07-3826651e4d3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:35:59.433468 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08402d36-d099-4363-8d07-3826651e4d3a admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:35:59.434180 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08402d36-d099-4363-8d07-3826651e4d3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:35:59.435735 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08402d36-d099-4363-8d07-3826651e4d3a admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:35:59.447595 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1314/2622] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:35:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 11:35:59.451644 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b853db27-aa00-426a-abfd-83b874d58329 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:35:59.461448 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b853db27-aa00-426a-abfd-83b874d58329 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:35:59.465947 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6bb87d-c3af-4dc5-8e78-38c8c8cd490c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:35:59.466809 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6bb87d-c3af-4dc5-8e78-38c8c8cd490c admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:35:59.467095 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6bb87d-c3af-4dc5-8e78-38c8c8cd490c admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:35:59.467373 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6bb87d-c3af-4dc5-8e78-38c8c8cd490c admin admin] PATH_INFO: `/v3/projects/1b9855bffa1445769dc2fd40849c3b1d/users/5fa5b36fad5440f3a87cbfcf52a7ec4c/roles/e1c88eebe911432cab6a07fbeb84653b` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:35:59.481556 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6bb87d-c3af-4dc5-8e78-38c8c8cd490c admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b9855bffa1445769dc2fd40849c3b1d, user_id=5fa5b36fad5440f3a87cbfcf52a7ec4c, role_id=e1c88eebe911432cab6a07fbeb84653b)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:35:59.484512 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6bb87d-c3af-4dc5-8e78-38c8c8cd490c admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:35:59.567753 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1309/2623] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jun 29 11:35:59 2022] PUT /identity/v3/projects/1b9855bffa1445769dc2fd40849c3b1d/users/5fa5b36fad5440f3a87cbfcf52a7ec4c/roles/e1c88eebe911432cab6a07fbeb84653b => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:35:59.578527 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511cd335-d5d6-4747-b5bf-839c7ff2494e None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:35:59.578878 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511cd335-d5d6-4747-b5bf-839c7ff2494e None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:35:59.579166 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511cd335-d5d6-4747-b5bf-839c7ff2494e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:35:59.605485 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-511cd335-d5d6-4747-b5bf-839c7ff2494e None None] MFA Rules not processed for user `5fa5b36fad5440f3a87cbfcf52a7ec4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:35:59.607376 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-511cd335-d5d6-4747-b5bf-839c7ff2494e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:35:59.672039 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1315/2624] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:35:59 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:35:59.683702 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6bb87d-c3af-4dc5-8e78-38c8c8cd490c admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:35:59.685412 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-ec6bb87d-c3af-4dc5-8e78-38c8c8cd490c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:35:59.741447 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6bb87d-c3af-4dc5-8e78-38c8c8cd490c admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:35:59.745042 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569b619c-7016-4d50-8b19-d125e0627363 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff7931b75a924394b1af733febce0834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:35:59.745733 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569b619c-7016-4d50-8b19-d125e0627363 service cinder] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:35:59.746018 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569b619c-7016-4d50-8b19-d125e0627363 service cinder] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:35:59.746298 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569b619c-7016-4d50-8b19-d125e0627363 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:35:59.754603 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569b619c-7016-4d50-8b19-d125e0627363 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:35:59.756060 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569b619c-7016-4d50-8b19-d125e0627363 service cinder] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:35:59.764745 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1310/2625] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:35:59 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:35:59.789280 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511cd335-d5d6-4747-b5bf-839c7ff2494e None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:35:59.851348 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511cd335-d5d6-4747-b5bf-839c7ff2494e None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:35:59.856071 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac7d711-c22a-4612-b14c-d9cd0f3994e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:35:59.857196 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac7d711-c22a-4612-b14c-d9cd0f3994e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:35:59.857481 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac7d711-c22a-4612-b14c-d9cd0f3994e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:35:59.857759 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac7d711-c22a-4612-b14c-d9cd0f3994e2 admin admin] PATH_INFO: `/v3/users/5fa5b36fad5440f3a87cbfcf52a7ec4c` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:35:59.858964 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac7d711-c22a-4612-b14c-d9cd0f3994e2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5fa5b36fad5440f3a87cbfcf52a7ec4c)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:35:59.860906 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac7d711-c22a-4612-b14c-d9cd0f3994e2 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:36:00.091011 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-9ac7d711-c22a-4612-b14c-d9cd0f3994e2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5fa5b36fad5440f3a87cbfcf52a7ec4c'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:36:00.091656 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-9ac7d711-c22a-4612-b14c-d9cd0f3994e2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5fa5b36fad5440f3a87cbfcf52a7ec4c'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:36:00.094178 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:36:00.197295 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-9ac7d711-c22a-4612-b14c-d9cd0f3994e2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5fa5b36fad5440f3a87cbfcf52a7ec4c'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:36:00.204193 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-9ac7d711-c22a-4612-b14c-d9cd0f3994e2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5fa5b36fad5440f3a87cbfcf52a7ec4c'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:36:00.204822 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-9ac7d711-c22a-4612-b14c-d9cd0f3994e2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5fa5b36fad5440f3a87cbfcf52a7ec4c'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:36:00.223006 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1316/2626] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jun 29 11:35:59 2022] DELETE /identity/v3/users/5fa5b36fad5440f3a87cbfcf52a7ec4c => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:36:00.227500 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e075db5-59cc-4f4b-9f22-0ae6294bbd6b None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:36:00.230193 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e075db5-59cc-4f4b-9f22-0ae6294bbd6b None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:36:00.230193 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e075db5-59cc-4f4b-9f22-0ae6294bbd6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:36:00.246307 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-8e075db5-59cc-4f4b-9f22-0ae6294bbd6b None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:36:00.247504 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-8e075db5-59cc-4f4b-9f22-0ae6294bbd6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:36:00.293406 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:36:00.305051 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1311/2627] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:36:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:36:00.314713 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac7d711-c22a-4612-b14c-d9cd0f3994e2 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:36:00.316778 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-9ac7d711-c22a-4612-b14c-d9cd0f3994e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:36:00.372644 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac7d711-c22a-4612-b14c-d9cd0f3994e2 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:36:00.376152 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb26c9c5-7ebb-46da-9a88-5e243f43544c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f4bb9f989e4024a5da28dced646b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:36:00.376799 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb26c9c5-7ebb-46da-9a88-5e243f43544c service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:36:00.377084 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb26c9c5-7ebb-46da-9a88-5e243f43544c service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:36:00.377965 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb26c9c5-7ebb-46da-9a88-5e243f43544c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:36:00.387093 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb26c9c5-7ebb-46da-9a88-5e243f43544c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:36:00.388551 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb26c9c5-7ebb-46da-9a88-5e243f43544c service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:36:00.397954 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1317/2628] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:36:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:36:00.818420 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e075db5-59cc-4f4b-9f22-0ae6294bbd6b None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:36:00.819870 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-8e075db5-59cc-4f4b-9f22-0ae6294bbd6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:36:00.832399 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e075db5-59cc-4f4b-9f22-0ae6294bbd6b None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:36:00.837081 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5101aea1-8e1a-46af-928b-1e3f3210a823 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:36:00.837946 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5101aea1-8e1a-46af-928b-1e3f3210a823 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:36:00.842370 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5101aea1-8e1a-46af-928b-1e3f3210a823 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:36:00.842370 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5101aea1-8e1a-46af-928b-1e3f3210a823 admin admin] PATH_INFO: `/v3/projects/1b9855bffa1445769dc2fd40849c3b1d` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:36:00.842370 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5101aea1-8e1a-46af-928b-1e3f3210a823 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1b9855bffa1445769dc2fd40849c3b1d)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:36:00.842370 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5101aea1-8e1a-46af-928b-1e3f3210a823 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:36:00.958111 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-5101aea1-8e1a-46af-928b-1e3f3210a823 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1b9855bffa1445769dc2fd40849c3b1d', 'request_id': 'req-5101aea1-8e1a-46af-928b-1e3f3210a823'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:36:01.042776 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-5101aea1-8e1a-46af-928b-1e3f3210a823 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1b9855bffa1445769dc2fd40849c3b1d', 'request_id': 'req-5101aea1-8e1a-46af-928b-1e3f3210a823'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:36:01.049088 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-5101aea1-8e1a-46af-928b-1e3f3210a823 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1b9855bffa1445769dc2fd40849c3b1d', 'request_id': 'req-5101aea1-8e1a-46af-928b-1e3f3210a823'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:36:01.059419 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-5101aea1-8e1a-46af-928b-1e3f3210a823 admin admin] The token cache is being invalidate because project 1b9855bffa1445769dc2fd40849c3b1d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 11:36:01.059707 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-5101aea1-8e1a-46af-928b-1e3f3210a823 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:36:01.060388 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-5101aea1-8e1a-46af-928b-1e3f3210a823 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:36:01.061916 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1312/2629] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jun 29 11:36:00 2022] DELETE /identity/v3/projects/1b9855bffa1445769dc2fd40849c3b1d => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:36:10.393727 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:DELETE=3,keystone:SELECT=38 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:36:11.056594 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=29,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:36:35.017954 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51aff9cb-eb98-4b73-a405-1825e906fa89 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:36:35.018534 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51aff9cb-eb98-4b73-a405-1825e906fa89 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:36:35.018534 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51aff9cb-eb98-4b73-a405-1825e906fa89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:36:35.036137 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-51aff9cb-eb98-4b73-a405-1825e906fa89 None None] MFA Rules not processed for user `cf7bc4681c954c158a7051046bd9aa47`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:36:35.038840 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-51aff9cb-eb98-4b73-a405-1825e906fa89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:36:35.093175 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1318/2630] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:36:35 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:36:35.400451 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5101aea1-8e1a-46af-928b-1e3f3210a823 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:36:35.401850 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-5101aea1-8e1a-46af-928b-1e3f3210a823 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:36:35.457822 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5101aea1-8e1a-46af-928b-1e3f3210a823 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:36:35.461929 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e506b0fb-2eec-44e1-a868-fb5bdf92a41d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f4bb9f989e4024a5da28dced646b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:36:35.462566 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e506b0fb-2eec-44e1-a868-fb5bdf92a41d service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:36:35.462848 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e506b0fb-2eec-44e1-a868-fb5bdf92a41d service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:36:35.463125 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e506b0fb-2eec-44e1-a868-fb5bdf92a41d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:36:35.464892 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-e506b0fb-2eec-44e1-a868-fb5bdf92a41d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:36:35.474137 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e506b0fb-2eec-44e1-a868-fb5bdf92a41d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:36:35.476170 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e506b0fb-2eec-44e1-a868-fb5bdf92a41d service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:36:35.484416 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1313/2631] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:36:35 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 11:36:35.906118 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51aff9cb-eb98-4b73-a405-1825e906fa89 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:36:35.907764 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-51aff9cb-eb98-4b73-a405-1825e906fa89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:36:35.986429 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51aff9cb-eb98-4b73-a405-1825e906fa89 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:36:35.992056 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccfedb8-c93b-46ba-9b3e-427c215abd33 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a0505342b704d02977a5125957c2c9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:36:35.992731 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccfedb8-c93b-46ba-9b3e-427c215abd33 service placement] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:36:35.993016 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccfedb8-c93b-46ba-9b3e-427c215abd33 service placement] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:36:35.993293 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccfedb8-c93b-46ba-9b3e-427c215abd33 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:36:35.995080 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-9ccfedb8-c93b-46ba-9b3e-427c215abd33 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:36:36.007067 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccfedb8-c93b-46ba-9b3e-427c215abd33 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:36:36.008675 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccfedb8-c93b-46ba-9b3e-427c215abd33 service placement] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:36:36.018749 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1319/2632] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:36:35 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:36:45.481150 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=26 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:36:46.014306 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=57 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:36:52.863173 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e506b0fb-2eec-44e1-a868-fb5bdf92a41d service neutron] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:36:52.864625 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-e506b0fb-2eec-44e1-a868-fb5bdf92a41d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:36:52.945104 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e506b0fb-2eec-44e1-a868-fb5bdf92a41d service neutron] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:36:52.949696 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4519b0-a7a5-4c92-abb1-e06d49901100 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '796aeeff16df4739bddec1f6dee61bfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:36:52.950357 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4519b0-a7a5-4c92-abb1-e06d49901100 service nova] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:36:52.950638 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4519b0-a7a5-4c92-abb1-e06d49901100 service nova] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:36:52.950916 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4519b0-a7a5-4c92-abb1-e06d49901100 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:36:52.953446 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-4e4519b0-a7a5-4c92-abb1-e06d49901100 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:36:52.970167 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4519b0-a7a5-4c92-abb1-e06d49901100 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:36:52.972302 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4519b0-a7a5-4c92-abb1-e06d49901100 service nova] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:36:52.986894 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1314/2633] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:36:52 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:36:57.052560 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccfedb8-c93b-46ba-9b3e-427c215abd33 service placement] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:36:57.062266 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccfedb8-c93b-46ba-9b3e-427c215abd33 service placement] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:36:57.065857 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6275f90e-5de5-46ec-a925-677296434d70 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f4bb9f989e4024a5da28dced646b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:36:57.066491 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6275f90e-5de5-46ec-a925-677296434d70 service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:36:57.066774 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6275f90e-5de5-46ec-a925-677296434d70 service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:36:57.067053 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6275f90e-5de5-46ec-a925-677296434d70 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:36:57.068860 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-6275f90e-5de5-46ec-a925-677296434d70 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:36:57.136228 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6275f90e-5de5-46ec-a925-677296434d70 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:36:57.137674 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6275f90e-5de5-46ec-a925-677296434d70 service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:36:57.147187 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1320/2634] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:36:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:37:00.845231 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4519b0-a7a5-4c92-abb1-e06d49901100 service nova] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:37:00.846731 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-4e4519b0-a7a5-4c92-abb1-e06d49901100 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:37:00.926463 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4519b0-a7a5-4c92-abb1-e06d49901100 service nova] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:37:00.931668 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f144d21a-5f87-425f-957a-5b3af0945116 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:37:00.932588 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f144d21a-5f87-425f-957a-5b3af0945116 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:37:00.932864 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f144d21a-5f87-425f-957a-5b3af0945116 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:37:00.933133 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f144d21a-5f87-425f-957a-5b3af0945116 admin admin] PATH_INFO: `/v3/users/275df17696084dce9e7af487275bacab` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:37:00.946350 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f144d21a-5f87-425f-957a-5b3af0945116 admin admin] RBAC: Authorizing `identity:delete_user(user_id=275df17696084dce9e7af487275bacab)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:37:00.948294 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f144d21a-5f87-425f-957a-5b3af0945116 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:37:01.124868 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-f144d21a-5f87-425f-957a-5b3af0945116 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '275df17696084dce9e7af487275bacab'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:37:01.208257 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-f144d21a-5f87-425f-957a-5b3af0945116 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '275df17696084dce9e7af487275bacab'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:37:01.208896 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-f144d21a-5f87-425f-957a-5b3af0945116 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '275df17696084dce9e7af487275bacab'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:37:01.219993 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:37:01.222166 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-f144d21a-5f87-425f-957a-5b3af0945116 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '275df17696084dce9e7af487275bacab'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:37:01.222787 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-f144d21a-5f87-425f-957a-5b3af0945116 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '275df17696084dce9e7af487275bacab'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:37:01.235246 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1315/2635] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jun 29 11:37:00 2022] DELETE /identity/v3/users/275df17696084dce9e7af487275bacab => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:37:01.463807 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6275f90e-5de5-46ec-a925-677296434d70 service neutron] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:37:01.518584 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6275f90e-5de5-46ec-a925-677296434d70 service neutron] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:37:01.573490 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc8161c-a7cf-47e8-8906-f6686ff8bb07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:37:01.574371 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc8161c-a7cf-47e8-8906-f6686ff8bb07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:37:01.574649 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc8161c-a7cf-47e8-8906-f6686ff8bb07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:37:01.574921 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc8161c-a7cf-47e8-8906-f6686ff8bb07 admin admin] PATH_INFO: `/v3/projects/364a7eaf772c434e9e38a18430425895` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:37:01.585308 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc8161c-a7cf-47e8-8906-f6686ff8bb07 admin admin] RBAC: Authorizing `identity:delete_project(project_id=364a7eaf772c434e9e38a18430425895)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:37:01.587184 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc8161c-a7cf-47e8-8906-f6686ff8bb07 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:37:01.672559 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:37:01.673888 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-2fc8161c-a7cf-47e8-8906-f6686ff8bb07 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '364a7eaf772c434e9e38a18430425895', 'request_id': 'req-2fc8161c-a7cf-47e8-8906-f6686ff8bb07'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:37:01.680522 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-2fc8161c-a7cf-47e8-8906-f6686ff8bb07 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '364a7eaf772c434e9e38a18430425895', 'request_id': 'req-2fc8161c-a7cf-47e8-8906-f6686ff8bb07'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:37:01.685993 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-2fc8161c-a7cf-47e8-8906-f6686ff8bb07 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '364a7eaf772c434e9e38a18430425895', 'request_id': 'req-2fc8161c-a7cf-47e8-8906-f6686ff8bb07'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:37:01.779616 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-2fc8161c-a7cf-47e8-8906-f6686ff8bb07 admin admin] The token cache is being invalidate because project 364a7eaf772c434e9e38a18430425895 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 11:37:01.779859 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-2fc8161c-a7cf-47e8-8906-f6686ff8bb07 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:37:01.780488 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-2fc8161c-a7cf-47e8-8906-f6686ff8bb07 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:37:01.782058 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1321/2636] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jun 29 11:37:01 2022] DELETE /identity/v3/projects/364a7eaf772c434e9e38a18430425895 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:37:01.787066 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c06021c-28e8-4110-a823-213bf976ff75 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:37:01.787358 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c06021c-28e8-4110-a823-213bf976ff75 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:37:01.787638 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c06021c-28e8-4110-a823-213bf976ff75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:37:01.833181 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-3c06021c-28e8-4110-a823-213bf976ff75 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:37:01.834472 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-3c06021c-28e8-4110-a823-213bf976ff75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:37:01.892128 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1316/2637] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:37:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:37:01.899613 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f80aeba-1bf3-497a-a64d-950fe751509e None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:37:01.899931 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f80aeba-1bf3-497a-a64d-950fe751509e None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:37:01.900212 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f80aeba-1bf3-497a-a64d-950fe751509e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:37:01.919038 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-8f80aeba-1bf3-497a-a64d-950fe751509e None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:37:01.920289 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-8f80aeba-1bf3-497a-a64d-950fe751509e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:37:01.928236 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1322/2638] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:37:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:37:01.932987 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c06021c-28e8-4110-a823-213bf976ff75 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:37:01.942848 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c06021c-28e8-4110-a823-213bf976ff75 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:37:01.947336 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c512426-b731-4eae-bda9-89c09c48c9f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:37:01.948026 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c512426-b731-4eae-bda9-89c09c48c9f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:37:01.948312 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c512426-b731-4eae-bda9-89c09c48c9f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:37:01.948586 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c512426-b731-4eae-bda9-89c09c48c9f7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:37:01.949306 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c512426-b731-4eae-bda9-89c09c48c9f7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:37:01.949617 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c512426-b731-4eae-bda9-89c09c48c9f7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:37:01.951205 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c512426-b731-4eae-bda9-89c09c48c9f7 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:37:01.962561 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1317/2639] 10.222.0.44 () {58 vars in 1152 bytes} [Wed Jun 29 11:37:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:37:01.966615 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f80aeba-1bf3-497a-a64d-950fe751509e None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:37:01.976422 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f80aeba-1bf3-497a-a64d-950fe751509e None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:37:01.981041 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045bf304-d656-4bf7-bb9b-0f3e8da71058 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:37:01.981684 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045bf304-d656-4bf7-bb9b-0f3e8da71058 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:37:01.981967 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045bf304-d656-4bf7-bb9b-0f3e8da71058 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:37:01.982241 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045bf304-d656-4bf7-bb9b-0f3e8da71058 admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:37:01.983202 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045bf304-d656-4bf7-bb9b-0f3e8da71058 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:37:01.985144 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045bf304-d656-4bf7-bb9b-0f3e8da71058 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:37:02.068710 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1323/2640] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jun 29 11:37:01 2022] POST /identity/v3/projects => generated 405 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:37:02.072715 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c512426-b731-4eae-bda9-89c09c48c9f7 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:37:02.082545 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c512426-b731-4eae-bda9-89c09c48c9f7 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:37:02.138062 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6491554a-c7e4-4c58-a298-ce91ed0cab85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:37:02.138770 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6491554a-c7e4-4c58-a298-ce91ed0cab85 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:37:02.139051 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6491554a-c7e4-4c58-a298-ce91ed0cab85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:37:02.139325 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6491554a-c7e4-4c58-a298-ce91ed0cab85 admin admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:37:02.140038 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6491554a-c7e4-4c58-a298-ce91ed0cab85 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:37:02.141945 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6491554a-c7e4-4c58-a298-ce91ed0cab85 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:37:02.271232 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1318/2641] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jun 29 11:37:02 2022] POST /identity/v3/users => generated 303 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:37:02.276029 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045bf304-d656-4bf7-bb9b-0f3e8da71058 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:37:02.286088 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045bf304-d656-4bf7-bb9b-0f3e8da71058 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:37:02.291306 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ad3d56-8da0-4590-939b-5c69edc4293c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:37:02.293460 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ad3d56-8da0-4590-939b-5c69edc4293c admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:37:02.293460 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ad3d56-8da0-4590-939b-5c69edc4293c admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:37:02.293460 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ad3d56-8da0-4590-939b-5c69edc4293c admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:37:02.293460 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ad3d56-8da0-4590-939b-5c69edc4293c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:37:02.294792 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ad3d56-8da0-4590-939b-5c69edc4293c admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:37:02.305016 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1324/2642] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:37:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 11:37:02.308818 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6491554a-c7e4-4c58-a298-ce91ed0cab85 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:37:02.318553 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6491554a-c7e4-4c58-a298-ce91ed0cab85 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:37:02.323259 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebef1a3-3a01-4041-8968-83386f0ba2c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:37:02.323864 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebef1a3-3a01-4041-8968-83386f0ba2c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:37:02.324152 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebef1a3-3a01-4041-8968-83386f0ba2c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:37:02.324428 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebef1a3-3a01-4041-8968-83386f0ba2c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:37:02.325148 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebef1a3-3a01-4041-8968-83386f0ba2c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:37:02.326728 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebef1a3-3a01-4041-8968-83386f0ba2c0 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:37:02.336822 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1319/2643] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:37:02 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 11:37:02.340837 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ad3d56-8da0-4590-939b-5c69edc4293c admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:37:02.351289 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ad3d56-8da0-4590-939b-5c69edc4293c admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:37:02.355786 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e493c52f-804f-4704-bdd3-b5de51c969a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:37:02.356681 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e493c52f-804f-4704-bdd3-b5de51c969a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:37:02.356965 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e493c52f-804f-4704-bdd3-b5de51c969a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:37:02.357240 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e493c52f-804f-4704-bdd3-b5de51c969a9 admin admin] PATH_INFO: `/v3/projects/a7af2ab925904cbb8127e1d86672acbc/users/9259589e0353490f92aa4f4866e47108/roles/e1c88eebe911432cab6a07fbeb84653b` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:37:02.372047 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e493c52f-804f-4704-bdd3-b5de51c969a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7af2ab925904cbb8127e1d86672acbc, user_id=9259589e0353490f92aa4f4866e47108, role_id=e1c88eebe911432cab6a07fbeb84653b)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:37:02.374671 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e493c52f-804f-4704-bdd3-b5de51c969a9 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:37:02.473264 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1325/2644] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jun 29 11:37:02 2022] PUT /identity/v3/projects/a7af2ab925904cbb8127e1d86672acbc/users/9259589e0353490f92aa4f4866e47108/roles/e1c88eebe911432cab6a07fbeb84653b => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 11:37:02.478919 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3af8989-fb33-4041-89b9-ec8ee7f35f3e None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:37:02.479211 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3af8989-fb33-4041-89b9-ec8ee7f35f3e None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:37:02.479492 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3af8989-fb33-4041-89b9-ec8ee7f35f3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:37:02.497238 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-d3af8989-fb33-4041-89b9-ec8ee7f35f3e None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:37:02.498436 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-d3af8989-fb33-4041-89b9-ec8ee7f35f3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:37:02.521988 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:37:02.556128 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1320/2645] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:37:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:37:02.565680 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e493c52f-804f-4704-bdd3-b5de51c969a9 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:37:02.567376 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-e493c52f-804f-4704-bdd3-b5de51c969a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:37:02.660971 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e493c52f-804f-4704-bdd3-b5de51c969a9 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:37:02.664570 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4076577-2a49-4ffb-9dd3-e3d42fbc14c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f4bb9f989e4024a5da28dced646b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:37:02.665217 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4076577-2a49-4ffb-9dd3-e3d42fbc14c2 service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:37:02.665505 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4076577-2a49-4ffb-9dd3-e3d42fbc14c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:37:02.665784 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4076577-2a49-4ffb-9dd3-e3d42fbc14c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:37:02.674707 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4076577-2a49-4ffb-9dd3-e3d42fbc14c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:37:02.676174 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4076577-2a49-4ffb-9dd3-e3d42fbc14c2 service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:37:02.685513 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1326/2646] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:37:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:37:03.894426 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438f52d3-7ff0-4188-856a-bdf43a86e86a None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:37:03.895750 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438f52d3-7ff0-4188-856a-bdf43a86e86a None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:37:03.895750 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438f52d3-7ff0-4188-856a-bdf43a86e86a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:37:03.926563 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-438f52d3-7ff0-4188-856a-bdf43a86e86a None None] MFA Rules not processed for user `9259589e0353490f92aa4f4866e47108`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:37:03.927778 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-438f52d3-7ff0-4188-856a-bdf43a86e86a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:37:03.995741 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1321/2647] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:37:03 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:37:04.000523 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4076577-2a49-4ffb-9dd3-e3d42fbc14c2 service neutron] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:37:04.010469 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4076577-2a49-4ffb-9dd3-e3d42fbc14c2 service neutron] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:37:04.015328 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72a99ab-7981-411d-b8cf-920a979e80d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:37:04.016031 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72a99ab-7981-411d-b8cf-920a979e80d7 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:37:04.016319 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72a99ab-7981-411d-b8cf-920a979e80d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:37:04.016599 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72a99ab-7981-411d-b8cf-920a979e80d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:37:04.017305 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72a99ab-7981-411d-b8cf-920a979e80d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:37:04.019495 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72a99ab-7981-411d-b8cf-920a979e80d7 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:37:04.112746 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1327/2648] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jun 29 11:37:03 2022] POST /identity/v3/projects => generated 403 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:37:04.116788 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438f52d3-7ff0-4188-856a-bdf43a86e86a None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:37:04.127337 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438f52d3-7ff0-4188-856a-bdf43a86e86a None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:37:04.181686 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9c958b-d4b5-4bba-b574-d67f3657a093 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:37:04.182339 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9c958b-d4b5-4bba-b574-d67f3657a093 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:37:04.182618 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9c958b-d4b5-4bba-b574-d67f3657a093 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:37:04.182892 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9c958b-d4b5-4bba-b574-d67f3657a093 admin admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:37:04.183581 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9c958b-d4b5-4bba-b574-d67f3657a093 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:37:04.185512 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9c958b-d4b5-4bba-b574-d67f3657a093 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:37:04.294075 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1322/2649] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jun 29 11:37:04 2022] POST /identity/v3/users => generated 308 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:37:04.297992 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72a99ab-7981-411d-b8cf-920a979e80d7 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:37:04.308259 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72a99ab-7981-411d-b8cf-920a979e80d7 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:37:04.312868 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8256bdfc-bee2-4067-90ae-cac6b24ae8d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:37:04.313522 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8256bdfc-bee2-4067-90ae-cac6b24ae8d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:37:04.313805 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8256bdfc-bee2-4067-90ae-cac6b24ae8d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:37:04.314084 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8256bdfc-bee2-4067-90ae-cac6b24ae8d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:37:04.314843 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8256bdfc-bee2-4067-90ae-cac6b24ae8d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:37:04.316474 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8256bdfc-bee2-4067-90ae-cac6b24ae8d0 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:37:04.326815 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1328/2650] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:37:04 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 11:37:04.330616 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9c958b-d4b5-4bba-b574-d67f3657a093 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:37:04.341510 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9c958b-d4b5-4bba-b574-d67f3657a093 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:37:04.346095 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b137c25a-ff4f-4876-b015-be5c80b8558f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:37:04.346953 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b137c25a-ff4f-4876-b015-be5c80b8558f admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:37:04.347256 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b137c25a-ff4f-4876-b015-be5c80b8558f admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:37:04.347504 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b137c25a-ff4f-4876-b015-be5c80b8558f admin admin] PATH_INFO: `/v3/projects/39e4d045848a4e3db408e0edd229ca3d/users/7ab4e21a6dbf429da3df729b7f262b82/roles/e95c2ce63e114cc19a9dda6ad21e794f` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:37:04.363223 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b137c25a-ff4f-4876-b015-be5c80b8558f admin admin] RBAC: Authorizing `identity:create_grant(project_id=39e4d045848a4e3db408e0edd229ca3d, user_id=7ab4e21a6dbf429da3df729b7f262b82, role_id=e95c2ce63e114cc19a9dda6ad21e794f)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:37:04.365862 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b137c25a-ff4f-4876-b015-be5c80b8558f admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:37:04.421180 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1323/2651] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jun 29 11:37:04 2022] PUT /identity/v3/projects/39e4d045848a4e3db408e0edd229ca3d/users/7ab4e21a6dbf429da3df729b7f262b82/roles/e95c2ce63e114cc19a9dda6ad21e794f => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 11:37:04.425057 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8256bdfc-bee2-4067-90ae-cac6b24ae8d0 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:37:04.435674 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8256bdfc-bee2-4067-90ae-cac6b24ae8d0 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:37:04.439279 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:37:04.492119 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662760d5-168d-4e07-9e0d-29b3c6c79a9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:37:04.492792 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662760d5-168d-4e07-9e0d-29b3c6c79a9b admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:37:04.493086 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662760d5-168d-4e07-9e0d-29b3c6c79a9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:37:04.493367 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662760d5-168d-4e07-9e0d-29b3c6c79a9b admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:37:04.494093 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662760d5-168d-4e07-9e0d-29b3c6c79a9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:37:04.495678 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662760d5-168d-4e07-9e0d-29b3c6c79a9b admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:37:04.505462 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1329/2652] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:37:04 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 11:37:04.510166 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b137c25a-ff4f-4876-b015-be5c80b8558f admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:37:04.520213 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b137c25a-ff4f-4876-b015-be5c80b8558f admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:37:04.524729 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34c6d56-e9e0-4346-ad48-97df5809e338 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:37:04.525609 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34c6d56-e9e0-4346-ad48-97df5809e338 admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:37:04.525890 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34c6d56-e9e0-4346-ad48-97df5809e338 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:37:04.526164 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34c6d56-e9e0-4346-ad48-97df5809e338 admin admin] PATH_INFO: `/v3/projects/39e4d045848a4e3db408e0edd229ca3d/users/7ab4e21a6dbf429da3df729b7f262b82/roles/e95c2ce63e114cc19a9dda6ad21e794f` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:37:04.528105 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34c6d56-e9e0-4346-ad48-97df5809e338 admin admin] RBAC: Authorizing `identity:create_grant(project_id=39e4d045848a4e3db408e0edd229ca3d, user_id=7ab4e21a6dbf429da3df729b7f262b82, role_id=e95c2ce63e114cc19a9dda6ad21e794f)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:37:04.530698 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34c6d56-e9e0-4346-ad48-97df5809e338 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:37:04.540603 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1324/2653] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jun 29 11:37:04 2022] PUT /identity/v3/projects/39e4d045848a4e3db408e0edd229ca3d/users/7ab4e21a6dbf429da3df729b7f262b82/roles/e95c2ce63e114cc19a9dda6ad21e794f => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:37:05.668633 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778466ce-6d8d-4154-9544-17d48f80fe26 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:37:05.669118 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778466ce-6d8d-4154-9544-17d48f80fe26 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:37:05.669248 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778466ce-6d8d-4154-9544-17d48f80fe26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:37:05.699000 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-778466ce-6d8d-4154-9544-17d48f80fe26 None None] MFA Rules not processed for user `7ab4e21a6dbf429da3df729b7f262b82`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:37:05.700280 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-778466ce-6d8d-4154-9544-17d48f80fe26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:37:05.770717 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1330/2654] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:37:05 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:37:06.914021 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34c6d56-e9e0-4346-ad48-97df5809e338 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:37:06.918572 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-e34c6d56-e9e0-4346-ad48-97df5809e338 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:37:06.930289 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:37:06.986523 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34c6d56-e9e0-4346-ad48-97df5809e338 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:37:06.991543 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c951b4-8cf2-4bba-946f-7bed1dd466e0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a0505342b704d02977a5125957c2c9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:37:06.992854 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c951b4-8cf2-4bba-946f-7bed1dd466e0 service placement] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:37:06.993135 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c951b4-8cf2-4bba-946f-7bed1dd466e0 service placement] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:37:06.993410 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c951b4-8cf2-4bba-946f-7bed1dd466e0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:37:06.995130 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-e3c951b4-8cf2-4bba-946f-7bed1dd466e0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:37:07.006116 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c951b4-8cf2-4bba-946f-7bed1dd466e0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:37:07.008317 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c951b4-8cf2-4bba-946f-7bed1dd466e0 service placement] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:37:07.018151 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1325/2655] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:37:06 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:37:07.034363 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778466ce-6d8d-4154-9544-17d48f80fe26 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:37:07.102302 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778466ce-6d8d-4154-9544-17d48f80fe26 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:37:07.106931 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd934aaf-8827-4dea-a514-1e00584b60ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:37:07.111623 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd934aaf-8827-4dea-a514-1e00584b60ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:37:07.111623 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd934aaf-8827-4dea-a514-1e00584b60ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:37:07.111623 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd934aaf-8827-4dea-a514-1e00584b60ea admin admin] PATH_INFO: `/v3/users/9259589e0353490f92aa4f4866e47108` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:37:07.111623 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd934aaf-8827-4dea-a514-1e00584b60ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=9259589e0353490f92aa4f4866e47108)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:37:07.112026 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd934aaf-8827-4dea-a514-1e00584b60ea admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:37:07.365297 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:37:07.378847 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-cd934aaf-8827-4dea-a514-1e00584b60ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9259589e0353490f92aa4f4866e47108'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:37:07.379502 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-cd934aaf-8827-4dea-a514-1e00584b60ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9259589e0353490f92aa4f4866e47108'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:37:07.472906 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-cd934aaf-8827-4dea-a514-1e00584b60ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9259589e0353490f92aa4f4866e47108'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:37:07.480952 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-cd934aaf-8827-4dea-a514-1e00584b60ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9259589e0353490f92aa4f4866e47108'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:37:07.481593 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-cd934aaf-8827-4dea-a514-1e00584b60ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9259589e0353490f92aa4f4866e47108'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:37:07.498485 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1331/2656] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jun 29 11:37:07 2022] DELETE /identity/v3/users/9259589e0353490f92aa4f4866e47108 => generated 0 bytes in 465 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:37:07.736274 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c951b4-8cf2-4bba-946f-7bed1dd466e0 service placement] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:37:07.751027 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c951b4-8cf2-4bba-946f-7bed1dd466e0 service placement] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:37:07.811782 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a07253e-40c2-4043-a845-eb878941854f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:37:07.812662 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a07253e-40c2-4043-a845-eb878941854f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:37:07.812990 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a07253e-40c2-4043-a845-eb878941854f admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:37:07.813279 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a07253e-40c2-4043-a845-eb878941854f admin admin] PATH_INFO: `/v3/projects/a7af2ab925904cbb8127e1d86672acbc` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:37:07.814476 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a07253e-40c2-4043-a845-eb878941854f admin admin] RBAC: Authorizing `identity:delete_project(project_id=a7af2ab925904cbb8127e1d86672acbc)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:37:07.816440 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a07253e-40c2-4043-a845-eb878941854f admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:37:07.957791 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-6a07253e-40c2-4043-a845-eb878941854f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a7af2ab925904cbb8127e1d86672acbc', 'request_id': 'req-6a07253e-40c2-4043-a845-eb878941854f'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:37:08.027513 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-6a07253e-40c2-4043-a845-eb878941854f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a7af2ab925904cbb8127e1d86672acbc', 'request_id': 'req-6a07253e-40c2-4043-a845-eb878941854f'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:37:08.034187 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-6a07253e-40c2-4043-a845-eb878941854f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a7af2ab925904cbb8127e1d86672acbc', 'request_id': 'req-6a07253e-40c2-4043-a845-eb878941854f'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:37:08.044574 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-6a07253e-40c2-4043-a845-eb878941854f admin admin] The token cache is being invalidate because project a7af2ab925904cbb8127e1d86672acbc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 11:37:08.044861 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-6a07253e-40c2-4043-a845-eb878941854f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:37:08.045511 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-6a07253e-40c2-4043-a845-eb878941854f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:37:08.047053 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1326/2657] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jun 29 11:37:07 2022] DELETE /identity/v3/projects/a7af2ab925904cbb8127e1d86672acbc => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:37:08.051279 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd934aaf-8827-4dea-a514-1e00584b60ea admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:37:08.053021 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-cd934aaf-8827-4dea-a514-1e00584b60ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:37:08.066097 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd934aaf-8827-4dea-a514-1e00584b60ea admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:37:08.123886 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c04a51-a413-4c76-9803-90a439b70812 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:37:08.124772 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c04a51-a413-4c76-9803-90a439b70812 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:37:08.125050 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c04a51-a413-4c76-9803-90a439b70812 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:37:08.125320 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c04a51-a413-4c76-9803-90a439b70812 admin admin] PATH_INFO: `/v3/users/7ab4e21a6dbf429da3df729b7f262b82` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:37:08.126502 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c04a51-a413-4c76-9803-90a439b70812 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ab4e21a6dbf429da3df729b7f262b82)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:37:08.128431 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c04a51-a413-4c76-9803-90a439b70812 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:37:08.281150 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-91c04a51-a413-4c76-9803-90a439b70812 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ab4e21a6dbf429da3df729b7f262b82'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:37:08.281803 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-91c04a51-a413-4c76-9803-90a439b70812 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ab4e21a6dbf429da3df729b7f262b82'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:37:08.330164 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-91c04a51-a413-4c76-9803-90a439b70812 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ab4e21a6dbf429da3df729b7f262b82'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:37:08.338017 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-91c04a51-a413-4c76-9803-90a439b70812 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ab4e21a6dbf429da3df729b7f262b82'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:37:08.340238 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-91c04a51-a413-4c76-9803-90a439b70812 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ab4e21a6dbf429da3df729b7f262b82'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:37:08.357510 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1332/2658] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jun 29 11:37:08 2022] DELETE /identity/v3/users/7ab4e21a6dbf429da3df729b7f262b82 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:37:08.624336 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a07253e-40c2-4043-a845-eb878941854f admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:37:08.625929 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-6a07253e-40c2-4043-a845-eb878941854f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:37:08.639951 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a07253e-40c2-4043-a845-eb878941854f admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:37:08.674867 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:37:08.703241 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9d9886-c8ce-4d43-9ad0-97bf1f422e1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:37:08.704180 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9d9886-c8ce-4d43-9ad0-97bf1f422e1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:37:08.704469 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9d9886-c8ce-4d43-9ad0-97bf1f422e1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:37:08.704743 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9d9886-c8ce-4d43-9ad0-97bf1f422e1a admin admin] PATH_INFO: `/v3/projects/39e4d045848a4e3db408e0edd229ca3d` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:37:08.705979 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9d9886-c8ce-4d43-9ad0-97bf1f422e1a admin admin] RBAC: Authorizing `identity:delete_project(project_id=39e4d045848a4e3db408e0edd229ca3d)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:37:08.707933 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9d9886-c8ce-4d43-9ad0-97bf1f422e1a admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:37:08.851113 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-9b9d9886-c8ce-4d43-9ad0-97bf1f422e1a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '39e4d045848a4e3db408e0edd229ca3d', 'request_id': 'req-9b9d9886-c8ce-4d43-9ad0-97bf1f422e1a'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:37:08.959252 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-9b9d9886-c8ce-4d43-9ad0-97bf1f422e1a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '39e4d045848a4e3db408e0edd229ca3d', 'request_id': 'req-9b9d9886-c8ce-4d43-9ad0-97bf1f422e1a'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:37:08.981479 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-9b9d9886-c8ce-4d43-9ad0-97bf1f422e1a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '39e4d045848a4e3db408e0edd229ca3d', 'request_id': 'req-9b9d9886-c8ce-4d43-9ad0-97bf1f422e1a'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:37:08.993121 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-9b9d9886-c8ce-4d43-9ad0-97bf1f422e1a admin admin] The token cache is being invalidate because project 39e4d045848a4e3db408e0edd229ca3d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 11:37:08.993280 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-9b9d9886-c8ce-4d43-9ad0-97bf1f422e1a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:37:08.994202 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-9b9d9886-c8ce-4d43-9ad0-97bf1f422e1a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:37:08.999274 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1327/2659] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jun 29 11:37:08 2022] DELETE /identity/v3/projects/39e4d045848a4e3db408e0edd229ca3d => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:37:09.001324 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d18039-59ed-4da8-a2b4-ef23be99280c None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:37:09.001530 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d18039-59ed-4da8-a2b4-ef23be99280c None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:37:09.001936 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d18039-59ed-4da8-a2b4-ef23be99280c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:37:09.021037 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-03d18039-59ed-4da8-a2b4-ef23be99280c None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:37:09.022330 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-03d18039-59ed-4da8-a2b4-ef23be99280c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:37:09.046525 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:37:09.096897 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1333/2660] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:37:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:37:09.105720 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e26eaf-610e-4ee6-9149-e1f1813ef207 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:37:09.105902 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e26eaf-610e-4ee6-9149-e1f1813ef207 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:37:09.106143 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e26eaf-610e-4ee6-9149-e1f1813ef207 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:37:09.126535 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-d2e26eaf-610e-4ee6-9149-e1f1813ef207 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:37:09.127924 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-d2e26eaf-610e-4ee6-9149-e1f1813ef207 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:37:09.147395 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1328/2661] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:37:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:37:09.152305 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d18039-59ed-4da8-a2b4-ef23be99280c None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:37:09.162535 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d18039-59ed-4da8-a2b4-ef23be99280c None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:37:09.167910 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909599c3-4557-424c-b26c-a37e2eb7c518 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:37:09.168525 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909599c3-4557-424c-b26c-a37e2eb7c518 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:37:09.168812 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909599c3-4557-424c-b26c-a37e2eb7c518 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:37:09.169089 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909599c3-4557-424c-b26c-a37e2eb7c518 admin admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:37:09.169820 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909599c3-4557-424c-b26c-a37e2eb7c518 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:37:09.170129 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909599c3-4557-424c-b26c-a37e2eb7c518 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:37:09.171862 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909599c3-4557-424c-b26c-a37e2eb7c518 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:37:09.184499 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1334/2662] 10.222.0.44 () {58 vars in 1152 bytes} [Wed Jun 29 11:37:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:37:09.188689 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e26eaf-610e-4ee6-9149-e1f1813ef207 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:37:09.198377 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e26eaf-610e-4ee6-9149-e1f1813ef207 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:37:09.203671 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be759f0-5c18-466d-aa02-b0f81dfdd39e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:37:09.204364 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be759f0-5c18-466d-aa02-b0f81dfdd39e admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:37:09.204648 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be759f0-5c18-466d-aa02-b0f81dfdd39e admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:37:09.204914 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be759f0-5c18-466d-aa02-b0f81dfdd39e admin admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:37:09.205635 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be759f0-5c18-466d-aa02-b0f81dfdd39e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:37:09.207738 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be759f0-5c18-466d-aa02-b0f81dfdd39e admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:37:09.309152 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1329/2663] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jun 29 11:37:09 2022] POST /identity/v3/projects => generated 381 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 11:37:09.313742 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909599c3-4557-424c-b26c-a37e2eb7c518 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:37:09.329098 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909599c3-4557-424c-b26c-a37e2eb7c518 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:37:09.408742 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48900147-7857-42d2-b034-dd46616af133 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:37:09.409595 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48900147-7857-42d2-b034-dd46616af133 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:37:09.409884 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48900147-7857-42d2-b034-dd46616af133 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:37:09.410162 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48900147-7857-42d2-b034-dd46616af133 admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:37:09.410876 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48900147-7857-42d2-b034-dd46616af133 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:37:09.412990 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48900147-7857-42d2-b034-dd46616af133 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:37:09.508853 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1335/2664] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jun 29 11:37:09 2022] POST /identity/v3/users => generated 291 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 11:37:09.513050 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be759f0-5c18-466d-aa02-b0f81dfdd39e admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:37:09.524680 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be759f0-5c18-466d-aa02-b0f81dfdd39e admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:37:09.529464 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aeabb07-c102-41b6-b4cd-a5d89c921131 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:37:09.530222 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aeabb07-c102-41b6-b4cd-a5d89c921131 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:37:09.530591 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aeabb07-c102-41b6-b4cd-a5d89c921131 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:37:09.530941 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aeabb07-c102-41b6-b4cd-a5d89c921131 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:37:09.531779 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aeabb07-c102-41b6-b4cd-a5d89c921131 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:37:09.533428 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aeabb07-c102-41b6-b4cd-a5d89c921131 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:37:09.544445 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1330/2665] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:37:09 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 11:37:09.548575 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48900147-7857-42d2-b034-dd46616af133 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:37:09.559573 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48900147-7857-42d2-b034-dd46616af133 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:37:09.564515 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfea98ae-f6fa-41a4-b108-f4dcae6b633c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:37:09.565241 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfea98ae-f6fa-41a4-b108-f4dcae6b633c admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:37:09.567032 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfea98ae-f6fa-41a4-b108-f4dcae6b633c admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:37:09.567032 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfea98ae-f6fa-41a4-b108-f4dcae6b633c admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:37:09.567032 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfea98ae-f6fa-41a4-b108-f4dcae6b633c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:37:09.568549 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfea98ae-f6fa-41a4-b108-f4dcae6b633c admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:37:09.580254 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1336/2666] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:37:09 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 11:37:09.584724 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aeabb07-c102-41b6-b4cd-a5d89c921131 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:37:09.597705 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aeabb07-c102-41b6-b4cd-a5d89c921131 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:37:09.602368 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b680d9fb-0644-4b2b-9b2c-f28cc9cb157c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:37:09.603267 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b680d9fb-0644-4b2b-9b2c-f28cc9cb157c admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:37:09.603540 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b680d9fb-0644-4b2b-9b2c-f28cc9cb157c admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:37:09.603820 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b680d9fb-0644-4b2b-9b2c-f28cc9cb157c admin admin] PATH_INFO: `/v3/projects/1038f87c2e954dd388449b6f27e52c10/users/fb0ea60c1d8a48b19d15d4c44d755306/roles/e1c88eebe911432cab6a07fbeb84653b` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:37:09.618717 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b680d9fb-0644-4b2b-9b2c-f28cc9cb157c admin admin] RBAC: Authorizing `identity:create_grant(project_id=1038f87c2e954dd388449b6f27e52c10, user_id=fb0ea60c1d8a48b19d15d4c44d755306, role_id=e1c88eebe911432cab6a07fbeb84653b)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:37:09.624642 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b680d9fb-0644-4b2b-9b2c-f28cc9cb157c admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:37:09.728974 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1331/2667] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jun 29 11:37:09 2022] PUT /identity/v3/projects/1038f87c2e954dd388449b6f27e52c10/users/fb0ea60c1d8a48b19d15d4c44d755306/roles/e1c88eebe911432cab6a07fbeb84653b => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:37:09.739248 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e224542f-d1e5-483b-a26d-722b8c1dbd5b None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:37:09.739542 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e224542f-d1e5-483b-a26d-722b8c1dbd5b None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:37:09.739849 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e224542f-d1e5-483b-a26d-722b8c1dbd5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:37:09.767454 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-e224542f-d1e5-483b-a26d-722b8c1dbd5b None None] MFA Rules not processed for user `fb0ea60c1d8a48b19d15d4c44d755306`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:37:09.768756 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-e224542f-d1e5-483b-a26d-722b8c1dbd5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:37:09.835143 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1337/2668] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:37:09 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:37:09.839751 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b680d9fb-0644-4b2b-9b2c-f28cc9cb157c admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:37:09.902145 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:37:09.906197 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b680d9fb-0644-4b2b-9b2c-f28cc9cb157c admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:37:09.911063 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d021f042-73aa-4c6a-bc09-ffea67200d78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:37:09.911779 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d021f042-73aa-4c6a-bc09-ffea67200d78 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:37:09.912091 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d021f042-73aa-4c6a-bc09-ffea67200d78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:37:09.912366 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d021f042-73aa-4c6a-bc09-ffea67200d78 admin admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:37:09.913102 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d021f042-73aa-4c6a-bc09-ffea67200d78 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:37:09.919161 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d021f042-73aa-4c6a-bc09-ffea67200d78 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:37:10.043457 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1332/2669] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jun 29 11:37:09 2022] POST /identity/v3/projects => generated 383 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:37:10.044872 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e224542f-d1e5-483b-a26d-722b8c1dbd5b None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:37:10.056874 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e224542f-d1e5-483b-a26d-722b8c1dbd5b None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:37:10.080617 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:37:10.118371 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bf9c8f-f7bb-4e79-a10d-134652cc9c95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:37:10.119100 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bf9c8f-f7bb-4e79-a10d-134652cc9c95 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:37:10.119403 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bf9c8f-f7bb-4e79-a10d-134652cc9c95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:37:10.119644 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bf9c8f-f7bb-4e79-a10d-134652cc9c95 admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:37:10.120348 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bf9c8f-f7bb-4e79-a10d-134652cc9c95 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:37:10.122271 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bf9c8f-f7bb-4e79-a10d-134652cc9c95 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:37:10.208457 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1338/2670] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jun 29 11:37:10 2022] POST /identity/v3/users => generated 298 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 11:37:10.212948 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d021f042-73aa-4c6a-bc09-ffea67200d78 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:37:10.223366 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d021f042-73aa-4c6a-bc09-ffea67200d78 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:37:10.228279 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6076bae0-afec-4915-8275-2db821c81b27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:37:10.228943 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6076bae0-afec-4915-8275-2db821c81b27 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:37:10.229228 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6076bae0-afec-4915-8275-2db821c81b27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:37:10.229506 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6076bae0-afec-4915-8275-2db821c81b27 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:37:10.230233 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6076bae0-afec-4915-8275-2db821c81b27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:37:10.231843 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6076bae0-afec-4915-8275-2db821c81b27 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:37:10.242653 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1333/2671] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:37:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 11:37:10.246571 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bf9c8f-f7bb-4e79-a10d-134652cc9c95 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:37:10.256596 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bf9c8f-f7bb-4e79-a10d-134652cc9c95 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:37:10.261626 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b75a2c-ac13-4013-abaf-a0fc3a22fd78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:37:10.262498 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b75a2c-ac13-4013-abaf-a0fc3a22fd78 admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:37:10.262809 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b75a2c-ac13-4013-abaf-a0fc3a22fd78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:37:10.263083 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b75a2c-ac13-4013-abaf-a0fc3a22fd78 admin admin] PATH_INFO: `/v3/projects/8a8c32e5dc704640acb0582fe6f4adeb/users/fbb631753a724af3a5036077e689b42f/roles/e95c2ce63e114cc19a9dda6ad21e794f` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:37:10.278259 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b75a2c-ac13-4013-abaf-a0fc3a22fd78 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a8c32e5dc704640acb0582fe6f4adeb, user_id=fbb631753a724af3a5036077e689b42f, role_id=e95c2ce63e114cc19a9dda6ad21e794f)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:37:10.280919 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b75a2c-ac13-4013-abaf-a0fc3a22fd78 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:37:10.345141 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1339/2672] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jun 29 11:37:10 2022] PUT /identity/v3/projects/8a8c32e5dc704640acb0582fe6f4adeb/users/fbb631753a724af3a5036077e689b42f/roles/e95c2ce63e114cc19a9dda6ad21e794f => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:37:10.349184 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6076bae0-afec-4915-8275-2db821c81b27 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:37:10.360026 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6076bae0-afec-4915-8275-2db821c81b27 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:37:10.418210 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f7cc8e-ac5e-49f1-9040-d6253ec74c4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:37:10.418871 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f7cc8e-ac5e-49f1-9040-d6253ec74c4e admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:37:10.419155 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f7cc8e-ac5e-49f1-9040-d6253ec74c4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:37:10.419430 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f7cc8e-ac5e-49f1-9040-d6253ec74c4e admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:37:10.420184 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f7cc8e-ac5e-49f1-9040-d6253ec74c4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:37:10.421779 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f7cc8e-ac5e-49f1-9040-d6253ec74c4e admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:37:10.431242 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1334/2673] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:37:10 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 11:37:10.435578 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b75a2c-ac13-4013-abaf-a0fc3a22fd78 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:37:10.449617 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b75a2c-ac13-4013-abaf-a0fc3a22fd78 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:37:10.454603 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb3f4dc-1688-4e45-957d-07b6f8dab6d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:37:10.455506 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb3f4dc-1688-4e45-957d-07b6f8dab6d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:37:10.455793 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb3f4dc-1688-4e45-957d-07b6f8dab6d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:37:10.456109 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb3f4dc-1688-4e45-957d-07b6f8dab6d7 admin admin] PATH_INFO: `/v3/projects/8a8c32e5dc704640acb0582fe6f4adeb/users/fbb631753a724af3a5036077e689b42f/roles/e95c2ce63e114cc19a9dda6ad21e794f` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:37:10.458344 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb3f4dc-1688-4e45-957d-07b6f8dab6d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a8c32e5dc704640acb0582fe6f4adeb, user_id=fbb631753a724af3a5036077e689b42f, role_id=e95c2ce63e114cc19a9dda6ad21e794f)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:37:10.461615 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb3f4dc-1688-4e45-957d-07b6f8dab6d7 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:37:10.472414 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1340/2674] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jun 29 11:37:10 2022] PUT /identity/v3/projects/8a8c32e5dc704640acb0582fe6f4adeb/users/fbb631753a724af3a5036077e689b42f/roles/e95c2ce63e114cc19a9dda6ad21e794f => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:37:10.485006 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40bbb4e-a933-42be-bdaf-7057d75d8c84 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:37:10.485298 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40bbb4e-a933-42be-bdaf-7057d75d8c84 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:37:10.485571 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40bbb4e-a933-42be-bdaf-7057d75d8c84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:37:10.517651 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-c40bbb4e-a933-42be-bdaf-7057d75d8c84 None None] MFA Rules not processed for user `fbb631753a724af3a5036077e689b42f`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:37:10.518868 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-c40bbb4e-a933-42be-bdaf-7057d75d8c84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:37:10.592584 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1335/2675] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:37:10 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:37:10.603596 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb3f4dc-1688-4e45-957d-07b6f8dab6d7 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:37:10.605408 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-7bb3f4dc-1688-4e45-957d-07b6f8dab6d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:37:10.667054 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb3f4dc-1688-4e45-957d-07b6f8dab6d7 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:37:10.670715 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1a415c-988f-4377-9c10-5a4e17d1acc2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff7931b75a924394b1af733febce0834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:37:10.671390 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1a415c-988f-4377-9c10-5a4e17d1acc2 service cinder] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:37:10.671676 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1a415c-988f-4377-9c10-5a4e17d1acc2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:37:10.671982 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1a415c-988f-4377-9c10-5a4e17d1acc2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:37:10.682064 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1a415c-988f-4377-9c10-5a4e17d1acc2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:37:10.683529 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1a415c-988f-4377-9c10-5a4e17d1acc2 service cinder] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:37:10.693635 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1341/2676] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:37:10 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:37:10.798823 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40bbb4e-a933-42be-bdaf-7057d75d8c84 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:37:10.809285 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40bbb4e-a933-42be-bdaf-7057d75d8c84 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:37:10.812943 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bc0950-5f45-4d44-a9bd-6825a525f836 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff7931b75a924394b1af733febce0834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:37:10.813578 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bc0950-5f45-4d44-a9bd-6825a525f836 service cinder] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:37:10.813862 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bc0950-5f45-4d44-a9bd-6825a525f836 service cinder] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:37:10.814139 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bc0950-5f45-4d44-a9bd-6825a525f836 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:37:10.862003 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:37:10.871606 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bc0950-5f45-4d44-a9bd-6825a525f836 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:37:10.873984 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bc0950-5f45-4d44-a9bd-6825a525f836 service cinder] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:37:10.882278 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1336/2677] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:37:10 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:37:14.996300 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1a415c-988f-4377-9c10-5a4e17d1acc2 service cinder] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:37:15.058002 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1a415c-988f-4377-9c10-5a4e17d1acc2 service cinder] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:37:15.062584 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9656b842-b322-4f87-a4ce-d2e97e0604b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:37:15.063441 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9656b842-b322-4f87-a4ce-d2e97e0604b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:37:15.063727 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9656b842-b322-4f87-a4ce-d2e97e0604b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:37:15.064037 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9656b842-b322-4f87-a4ce-d2e97e0604b3 admin admin] PATH_INFO: `/v3/users/fb0ea60c1d8a48b19d15d4c44d755306` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:37:15.065212 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9656b842-b322-4f87-a4ce-d2e97e0604b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb0ea60c1d8a48b19d15d4c44d755306)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:37:15.067116 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9656b842-b322-4f87-a4ce-d2e97e0604b3 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:37:15.087774 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=93,keystone:INSERT=5,keystone:DELETE=2 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:37:15.269488 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-9656b842-b322-4f87-a4ce-d2e97e0604b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb0ea60c1d8a48b19d15d4c44d755306'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:37:15.270205 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-9656b842-b322-4f87-a4ce-d2e97e0604b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb0ea60c1d8a48b19d15d4c44d755306'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:37:15.335786 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-9656b842-b322-4f87-a4ce-d2e97e0604b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb0ea60c1d8a48b19d15d4c44d755306'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:37:15.343109 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-9656b842-b322-4f87-a4ce-d2e97e0604b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb0ea60c1d8a48b19d15d4c44d755306'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:37:15.344349 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-9656b842-b322-4f87-a4ce-d2e97e0604b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb0ea60c1d8a48b19d15d4c44d755306'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:37:15.361689 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1342/2678] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jun 29 11:37:14 2022] DELETE /identity/v3/users/fb0ea60c1d8a48b19d15d4c44d755306 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:37:15.366629 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0219dd72-f307-433a-92af-29a4150fce97 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:37:15.367106 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0219dd72-f307-433a-92af-29a4150fce97 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:37:15.367267 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0219dd72-f307-433a-92af-29a4150fce97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:37:15.385463 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-0219dd72-f307-433a-92af-29a4150fce97 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:37:15.386631 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-0219dd72-f307-433a-92af-29a4150fce97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:37:15.445314 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1337/2679] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:37:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:37:15.454863 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9656b842-b322-4f87-a4ce-d2e97e0604b3 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:37:15.456302 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-9656b842-b322-4f87-a4ce-d2e97e0604b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:37:15.519874 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9656b842-b322-4f87-a4ce-d2e97e0604b3 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:37:15.523425 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edbedf9-47e7-4669-b4d0-bc2c31b14428 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f4bb9f989e4024a5da28dced646b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:37:15.524132 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edbedf9-47e7-4669-b4d0-bc2c31b14428 service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:37:15.524413 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edbedf9-47e7-4669-b4d0-bc2c31b14428 service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:37:15.524683 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edbedf9-47e7-4669-b4d0-bc2c31b14428 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:37:15.533532 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edbedf9-47e7-4669-b4d0-bc2c31b14428 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:37:15.534942 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edbedf9-47e7-4669-b4d0-bc2c31b14428 service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:37:15.544787 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1343/2680] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:37:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:37:15.972341 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0219dd72-f307-433a-92af-29a4150fce97 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:37:15.973202 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-0219dd72-f307-433a-92af-29a4150fce97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:37:15.986467 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0219dd72-f307-433a-92af-29a4150fce97 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:37:15.991129 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc1b590-8806-4a52-893e-2c25f17e689b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:37:15.992045 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc1b590-8806-4a52-893e-2c25f17e689b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:37:15.992322 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc1b590-8806-4a52-893e-2c25f17e689b admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:37:15.992613 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc1b590-8806-4a52-893e-2c25f17e689b admin admin] PATH_INFO: `/v3/projects/1038f87c2e954dd388449b6f27e52c10` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:37:15.993817 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc1b590-8806-4a52-893e-2c25f17e689b admin admin] RBAC: Authorizing `identity:delete_project(project_id=1038f87c2e954dd388449b6f27e52c10)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:37:15.995889 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc1b590-8806-4a52-893e-2c25f17e689b admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:37:16.128060 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-8cc1b590-8806-4a52-893e-2c25f17e689b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1038f87c2e954dd388449b6f27e52c10', 'request_id': 'req-8cc1b590-8806-4a52-893e-2c25f17e689b'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:37:16.196890 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-8cc1b590-8806-4a52-893e-2c25f17e689b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1038f87c2e954dd388449b6f27e52c10', 'request_id': 'req-8cc1b590-8806-4a52-893e-2c25f17e689b'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:37:16.203284 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-8cc1b590-8806-4a52-893e-2c25f17e689b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1038f87c2e954dd388449b6f27e52c10', 'request_id': 'req-8cc1b590-8806-4a52-893e-2c25f17e689b'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:37:16.214152 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-8cc1b590-8806-4a52-893e-2c25f17e689b admin admin] The token cache is being invalidate because project 1038f87c2e954dd388449b6f27e52c10 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 11:37:16.214439 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-8cc1b590-8806-4a52-893e-2c25f17e689b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:37:16.215059 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-8cc1b590-8806-4a52-893e-2c25f17e689b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:37:16.216618 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1338/2681] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jun 29 11:37:15 2022] DELETE /identity/v3/projects/1038f87c2e954dd388449b6f27e52c10 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:37:16.220662 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edbedf9-47e7-4669-b4d0-bc2c31b14428 service neutron] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:37:16.230124 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edbedf9-47e7-4669-b4d0-bc2c31b14428 service neutron] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:37:16.285368 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f06a442-d180-4174-a978-d52d07989cc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:37:16.286223 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f06a442-d180-4174-a978-d52d07989cc4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:37:16.286505 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f06a442-d180-4174-a978-d52d07989cc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:37:16.286792 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f06a442-d180-4174-a978-d52d07989cc4 admin admin] PATH_INFO: `/v3/users/fbb631753a724af3a5036077e689b42f` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:37:16.288048 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f06a442-d180-4174-a978-d52d07989cc4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fbb631753a724af3a5036077e689b42f)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:37:16.290395 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f06a442-d180-4174-a978-d52d07989cc4 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:37:16.470564 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:37:16.477001 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-2f06a442-d180-4174-a978-d52d07989cc4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fbb631753a724af3a5036077e689b42f'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:37:16.477666 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-2f06a442-d180-4174-a978-d52d07989cc4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fbb631753a724af3a5036077e689b42f'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:37:16.552707 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-2f06a442-d180-4174-a978-d52d07989cc4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fbb631753a724af3a5036077e689b42f'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:37:16.561544 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-2f06a442-d180-4174-a978-d52d07989cc4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fbb631753a724af3a5036077e689b42f'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:37:16.562523 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-2f06a442-d180-4174-a978-d52d07989cc4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fbb631753a724af3a5036077e689b42f'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:37:16.580605 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1344/2682] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jun 29 11:37:16 2022] DELETE /identity/v3/users/fbb631753a724af3a5036077e689b42f => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:37:17.060063 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc1b590-8806-4a52-893e-2c25f17e689b admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:37:17.067111 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-8cc1b590-8806-4a52-893e-2c25f17e689b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:37:17.076285 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc1b590-8806-4a52-893e-2c25f17e689b admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:37:17.139891 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55942cf5-f2d4-4e38-a368-3b75dc9ce2cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:37:17.140786 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55942cf5-f2d4-4e38-a368-3b75dc9ce2cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:37:17.141071 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55942cf5-f2d4-4e38-a368-3b75dc9ce2cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:37:17.141348 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55942cf5-f2d4-4e38-a368-3b75dc9ce2cc admin admin] PATH_INFO: `/v3/projects/8a8c32e5dc704640acb0582fe6f4adeb` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:37:17.142579 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55942cf5-f2d4-4e38-a368-3b75dc9ce2cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a8c32e5dc704640acb0582fe6f4adeb)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:37:17.144558 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55942cf5-f2d4-4e38-a368-3b75dc9ce2cc admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:37:17.162800 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:37:17.286201 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-55942cf5-f2d4-4e38-a368-3b75dc9ce2cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a8c32e5dc704640acb0582fe6f4adeb', 'request_id': 'req-55942cf5-f2d4-4e38-a368-3b75dc9ce2cc'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:37:17.370310 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-55942cf5-f2d4-4e38-a368-3b75dc9ce2cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a8c32e5dc704640acb0582fe6f4adeb', 'request_id': 'req-55942cf5-f2d4-4e38-a368-3b75dc9ce2cc'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:37:17.379426 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-55942cf5-f2d4-4e38-a368-3b75dc9ce2cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a8c32e5dc704640acb0582fe6f4adeb', 'request_id': 'req-55942cf5-f2d4-4e38-a368-3b75dc9ce2cc'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:37:17.393180 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-55942cf5-f2d4-4e38-a368-3b75dc9ce2cc admin admin] The token cache is being invalidate because project 8a8c32e5dc704640acb0582fe6f4adeb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 11:37:17.393584 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-55942cf5-f2d4-4e38-a368-3b75dc9ce2cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:37:17.394395 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-55942cf5-f2d4-4e38-a368-3b75dc9ce2cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:37:17.396734 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1339/2683] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jun 29 11:37:17 2022] DELETE /identity/v3/projects/8a8c32e5dc704640acb0582fe6f4adeb => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:37:17.402667 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d7ce54-0e4b-4115-acc8-a171105c2565 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:37:17.403127 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d7ce54-0e4b-4115-acc8-a171105c2565 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:37:17.403230 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d7ce54-0e4b-4115-acc8-a171105c2565 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:37:17.445071 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-79d7ce54-0e4b-4115-acc8-a171105c2565 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:37:17.445917 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-79d7ce54-0e4b-4115-acc8-a171105c2565 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:37:17.518275 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1345/2684] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:37:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:37:17.526290 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2dafd1-7f2e-44ca-9c16-938658b38ea5 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:37:17.526709 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2dafd1-7f2e-44ca-9c16-938658b38ea5 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:37:17.527121 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2dafd1-7f2e-44ca-9c16-938658b38ea5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:37:17.552262 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-fa2dafd1-7f2e-44ca-9c16-938658b38ea5 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:37:17.561710 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-fa2dafd1-7f2e-44ca-9c16-938658b38ea5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:37:17.562888 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1340/2685] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:37:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:37:17.567076 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d7ce54-0e4b-4115-acc8-a171105c2565 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:37:17.579099 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d7ce54-0e4b-4115-acc8-a171105c2565 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:37:17.584588 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa11e32-be0e-417a-a7f4-d3f450d29067 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:37:17.590993 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa11e32-be0e-417a-a7f4-d3f450d29067 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:37:17.590993 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa11e32-be0e-417a-a7f4-d3f450d29067 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:37:17.590993 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa11e32-be0e-417a-a7f4-d3f450d29067 admin admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:37:17.590993 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa11e32-be0e-417a-a7f4-d3f450d29067 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:37:17.590993 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa11e32-be0e-417a-a7f4-d3f450d29067 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:37:17.590993 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa11e32-be0e-417a-a7f4-d3f450d29067 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:37:17.606290 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1346/2686] 10.222.0.44 () {58 vars in 1152 bytes} [Wed Jun 29 11:37:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:37:17.609057 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2dafd1-7f2e-44ca-9c16-938658b38ea5 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:37:17.618689 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2dafd1-7f2e-44ca-9c16-938658b38ea5 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:37:17.623617 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189e1f0f-e8bf-4eab-be3f-59def564c626 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:37:17.624437 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189e1f0f-e8bf-4eab-be3f-59def564c626 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:37:17.624827 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189e1f0f-e8bf-4eab-be3f-59def564c626 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:37:17.625207 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189e1f0f-e8bf-4eab-be3f-59def564c626 admin admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:37:17.626022 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189e1f0f-e8bf-4eab-be3f-59def564c626 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:37:17.628067 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189e1f0f-e8bf-4eab-be3f-59def564c626 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:37:17.739645 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1341/2687] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jun 29 11:37:17 2022] POST /identity/v3/projects => generated 365 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:37:17.741031 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa11e32-be0e-417a-a7f4-d3f450d29067 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:37:17.754898 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa11e32-be0e-417a-a7f4-d3f450d29067 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:37:17.824661 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe370eba-548d-431a-835c-4eafe5ed2a4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:37:17.825584 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe370eba-548d-431a-835c-4eafe5ed2a4a admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:37:17.825944 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe370eba-548d-431a-835c-4eafe5ed2a4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:37:17.826287 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe370eba-548d-431a-835c-4eafe5ed2a4a admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:37:17.827060 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe370eba-548d-431a-835c-4eafe5ed2a4a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:37:17.829305 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe370eba-548d-431a-835c-4eafe5ed2a4a admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:37:17.970721 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1347/2688] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jun 29 11:37:17 2022] POST /identity/v3/users => generated 283 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:37:17.974793 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189e1f0f-e8bf-4eab-be3f-59def564c626 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:37:17.985447 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189e1f0f-e8bf-4eab-be3f-59def564c626 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:37:17.990111 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d6807b-c4db-40a6-9fe6-80c68cbf40f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:37:17.990831 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d6807b-c4db-40a6-9fe6-80c68cbf40f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:37:17.991181 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d6807b-c4db-40a6-9fe6-80c68cbf40f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:37:17.991609 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d6807b-c4db-40a6-9fe6-80c68cbf40f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:37:17.993076 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d6807b-c4db-40a6-9fe6-80c68cbf40f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:37:17.994785 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d6807b-c4db-40a6-9fe6-80c68cbf40f8 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:37:18.005750 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1342/2689] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:37:17 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 11:37:18.010467 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe370eba-548d-431a-835c-4eafe5ed2a4a admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:37:18.020627 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe370eba-548d-431a-835c-4eafe5ed2a4a admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:37:18.025300 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723c3e60-8b3f-4b76-9730-4fc9823f3a83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:37:18.025954 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723c3e60-8b3f-4b76-9730-4fc9823f3a83 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:37:18.026237 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723c3e60-8b3f-4b76-9730-4fc9823f3a83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:37:18.026544 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723c3e60-8b3f-4b76-9730-4fc9823f3a83 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:37:18.027267 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723c3e60-8b3f-4b76-9730-4fc9823f3a83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:37:18.028944 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723c3e60-8b3f-4b76-9730-4fc9823f3a83 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:37:18.039649 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1348/2690] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:37:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 11:37:18.048967 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d6807b-c4db-40a6-9fe6-80c68cbf40f8 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:37:18.054761 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d6807b-c4db-40a6-9fe6-80c68cbf40f8 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:37:18.059883 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499615be-f06c-44be-9179-b7ccb9ad9fee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:37:18.060775 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499615be-f06c-44be-9179-b7ccb9ad9fee admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:37:18.061329 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499615be-f06c-44be-9179-b7ccb9ad9fee admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:37:18.061329 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499615be-f06c-44be-9179-b7ccb9ad9fee admin admin] PATH_INFO: `/v3/projects/d7c0e0b9977c4b29bb87200870d8357d/users/784378f7f2714fe0969344d26b871a9d/roles/e1c88eebe911432cab6a07fbeb84653b` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:37:18.078409 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499615be-f06c-44be-9179-b7ccb9ad9fee admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7c0e0b9977c4b29bb87200870d8357d, user_id=784378f7f2714fe0969344d26b871a9d, role_id=e1c88eebe911432cab6a07fbeb84653b)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:37:18.081125 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499615be-f06c-44be-9179-b7ccb9ad9fee admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:37:18.204042 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1343/2691] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jun 29 11:37:18 2022] PUT /identity/v3/projects/d7c0e0b9977c4b29bb87200870d8357d/users/784378f7f2714fe0969344d26b871a9d/roles/e1c88eebe911432cab6a07fbeb84653b => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:37:18.214401 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd3e66c-747a-4a18-8cda-e7948e925fb2 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:37:18.215565 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd3e66c-747a-4a18-8cda-e7948e925fb2 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:37:18.215565 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd3e66c-747a-4a18-8cda-e7948e925fb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:37:18.245017 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-0bd3e66c-747a-4a18-8cda-e7948e925fb2 None None] MFA Rules not processed for user `784378f7f2714fe0969344d26b871a9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:37:18.246266 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-0bd3e66c-747a-4a18-8cda-e7948e925fb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:37:18.250503 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:37:18.318194 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1349/2692] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:37:18 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 11:37:18.329908 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499615be-f06c-44be-9179-b7ccb9ad9fee admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:37:18.331723 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-499615be-f06c-44be-9179-b7ccb9ad9fee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:37:18.394959 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499615be-f06c-44be-9179-b7ccb9ad9fee admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:37:18.398933 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9806a3-84f8-41df-b900-c0334be261e4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff7931b75a924394b1af733febce0834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:37:18.399649 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9806a3-84f8-41df-b900-c0334be261e4 service cinder] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:37:18.400026 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9806a3-84f8-41df-b900-c0334be261e4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:37:18.400378 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9806a3-84f8-41df-b900-c0334be261e4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:37:18.410070 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9806a3-84f8-41df-b900-c0334be261e4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:37:18.411681 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9806a3-84f8-41df-b900-c0334be261e4 service cinder] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:37:18.424901 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1344/2693] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:37:18 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:37:23.642081 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd3e66c-747a-4a18-8cda-e7948e925fb2 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:37:23.643579 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-0bd3e66c-747a-4a18-8cda-e7948e925fb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:37:23.701903 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd3e66c-747a-4a18-8cda-e7948e925fb2 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:37:23.705514 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1eb80e-e14a-4299-acee-9f6e6fc19c8a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f4bb9f989e4024a5da28dced646b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:37:23.706165 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1eb80e-e14a-4299-acee-9f6e6fc19c8a service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:37:23.706452 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1eb80e-e14a-4299-acee-9f6e6fc19c8a service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:37:23.706734 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1eb80e-e14a-4299-acee-9f6e6fc19c8a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:37:23.708570 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-8a1eb80e-e14a-4299-acee-9f6e6fc19c8a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:37:23.779364 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1eb80e-e14a-4299-acee-9f6e6fc19c8a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:37:23.781654 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1eb80e-e14a-4299-acee-9f6e6fc19c8a service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:37:23.794197 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1350/2694] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:37:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:37:28.419656 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=70,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:37:31.410128 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9806a3-84f8-41df-b900-c0334be261e4 service cinder] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:37:31.411566 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-6a9806a3-84f8-41df-b900-c0334be261e4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:37:31.426590 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9806a3-84f8-41df-b900-c0334be261e4 service cinder] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:37:31.431113 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fce2be5-7806-49f4-8613-d2937c26c7c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:37:31.432586 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fce2be5-7806-49f4-8613-d2937c26c7c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:37:31.432868 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fce2be5-7806-49f4-8613-d2937c26c7c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:37:31.433142 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fce2be5-7806-49f4-8613-d2937c26c7c2 admin admin] PATH_INFO: `/v3/users/e91cc0b93f484f759233469ec5f37bb4` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:37:31.434327 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fce2be5-7806-49f4-8613-d2937c26c7c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e91cc0b93f484f759233469ec5f37bb4)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:37:31.436351 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fce2be5-7806-49f4-8613-d2937c26c7c2 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:37:31.588775 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-1fce2be5-7806-49f4-8613-d2937c26c7c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e91cc0b93f484f759233469ec5f37bb4'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:37:31.665454 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-1fce2be5-7806-49f4-8613-d2937c26c7c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e91cc0b93f484f759233469ec5f37bb4'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:37:31.666605 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-1fce2be5-7806-49f4-8613-d2937c26c7c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e91cc0b93f484f759233469ec5f37bb4'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:37:31.678880 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-1fce2be5-7806-49f4-8613-d2937c26c7c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e91cc0b93f484f759233469ec5f37bb4'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:37:31.679500 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-1fce2be5-7806-49f4-8613-d2937c26c7c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e91cc0b93f484f759233469ec5f37bb4'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:37:31.692827 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1345/2695] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jun 29 11:37:31 2022] DELETE /identity/v3/users/e91cc0b93f484f759233469ec5f37bb4 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:37:31.931025 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1eb80e-e14a-4299-acee-9f6e6fc19c8a service neutron] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:37:31.942225 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1eb80e-e14a-4299-acee-9f6e6fc19c8a service neutron] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:37:31.993895 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:37:32.000506 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778b6eff-bad2-4f16-977f-674ccc6a9677 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:37:32.001378 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778b6eff-bad2-4f16-977f-674ccc6a9677 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:37:32.002380 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778b6eff-bad2-4f16-977f-674ccc6a9677 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:37:32.002380 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778b6eff-bad2-4f16-977f-674ccc6a9677 admin admin] PATH_INFO: `/v3/projects/cdc0456bfa5a4619a96b41d3437fd82c` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:37:32.003212 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778b6eff-bad2-4f16-977f-674ccc6a9677 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cdc0456bfa5a4619a96b41d3437fd82c)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:37:32.005204 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778b6eff-bad2-4f16-977f-674ccc6a9677 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:37:32.135484 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-778b6eff-bad2-4f16-977f-674ccc6a9677 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cdc0456bfa5a4619a96b41d3437fd82c', 'request_id': 'req-778b6eff-bad2-4f16-977f-674ccc6a9677'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:37:32.141939 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-778b6eff-bad2-4f16-977f-674ccc6a9677 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cdc0456bfa5a4619a96b41d3437fd82c', 'request_id': 'req-778b6eff-bad2-4f16-977f-674ccc6a9677'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:37:32.147407 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-778b6eff-bad2-4f16-977f-674ccc6a9677 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cdc0456bfa5a4619a96b41d3437fd82c', 'request_id': 'req-778b6eff-bad2-4f16-977f-674ccc6a9677'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:37:32.224423 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-778b6eff-bad2-4f16-977f-674ccc6a9677 admin admin] The token cache is being invalidate because project cdc0456bfa5a4619a96b41d3437fd82c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 11:37:32.224714 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-778b6eff-bad2-4f16-977f-674ccc6a9677 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:37:32.225324 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-778b6eff-bad2-4f16-977f-674ccc6a9677 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:37:32.226840 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1351/2696] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jun 29 11:37:31 2022] DELETE /identity/v3/projects/cdc0456bfa5a4619a96b41d3437fd82c => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:37:32.231384 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fce2be5-7806-49f4-8613-d2937c26c7c2 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:37:32.232814 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-1fce2be5-7806-49f4-8613-d2937c26c7c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:37:32.246692 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fce2be5-7806-49f4-8613-d2937c26c7c2 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:37:32.302164 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33815d38-64b2-4911-9279-ca3843b67aa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:37:32.303054 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33815d38-64b2-4911-9279-ca3843b67aa2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:37:32.303341 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33815d38-64b2-4911-9279-ca3843b67aa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:37:32.303619 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33815d38-64b2-4911-9279-ca3843b67aa2 admin admin] PATH_INFO: `/v3/users/c49d176010fe49c38c7efc25926da2a3` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:37:32.304829 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33815d38-64b2-4911-9279-ca3843b67aa2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c49d176010fe49c38c7efc25926da2a3)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:37:32.307359 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33815d38-64b2-4911-9279-ca3843b67aa2 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:37:32.503996 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-33815d38-64b2-4911-9279-ca3843b67aa2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c49d176010fe49c38c7efc25926da2a3'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:37:32.581089 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-33815d38-64b2-4911-9279-ca3843b67aa2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c49d176010fe49c38c7efc25926da2a3'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:37:32.581706 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-33815d38-64b2-4911-9279-ca3843b67aa2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c49d176010fe49c38c7efc25926da2a3'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:37:32.594131 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-33815d38-64b2-4911-9279-ca3843b67aa2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c49d176010fe49c38c7efc25926da2a3'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:37:32.594756 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-33815d38-64b2-4911-9279-ca3843b67aa2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c49d176010fe49c38c7efc25926da2a3'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:37:32.596724 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:37:32.607468 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1346/2697] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jun 29 11:37:32 2022] DELETE /identity/v3/users/c49d176010fe49c38c7efc25926da2a3 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:37:32.857784 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778b6eff-bad2-4f16-977f-674ccc6a9677 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:37:32.859565 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-778b6eff-bad2-4f16-977f-674ccc6a9677 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:37:32.874846 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778b6eff-bad2-4f16-977f-674ccc6a9677 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:37:32.939037 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0ff572-0f09-4cd3-bb92-deda90c070a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:37:32.940502 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0ff572-0f09-4cd3-bb92-deda90c070a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:37:32.940890 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0ff572-0f09-4cd3-bb92-deda90c070a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:37:32.941180 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0ff572-0f09-4cd3-bb92-deda90c070a8 admin admin] PATH_INFO: `/v3/projects/7d0081550eb44279a4275e57d92f65e3` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:37:32.942441 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0ff572-0f09-4cd3-bb92-deda90c070a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d0081550eb44279a4275e57d92f65e3)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:37:32.944389 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0ff572-0f09-4cd3-bb92-deda90c070a8 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:37:33.141712 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-fe0ff572-0f09-4cd3-bb92-deda90c070a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d0081550eb44279a4275e57d92f65e3', 'request_id': 'req-fe0ff572-0f09-4cd3-bb92-deda90c070a8'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:37:33.147742 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-fe0ff572-0f09-4cd3-bb92-deda90c070a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d0081550eb44279a4275e57d92f65e3', 'request_id': 'req-fe0ff572-0f09-4cd3-bb92-deda90c070a8'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:37:33.153786 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-fe0ff572-0f09-4cd3-bb92-deda90c070a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d0081550eb44279a4275e57d92f65e3', 'request_id': 'req-fe0ff572-0f09-4cd3-bb92-deda90c070a8'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:37:33.242827 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-fe0ff572-0f09-4cd3-bb92-deda90c070a8 admin admin] The token cache is being invalidate because project 7d0081550eb44279a4275e57d92f65e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 11:37:33.243117 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-fe0ff572-0f09-4cd3-bb92-deda90c070a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:37:33.243758 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-fe0ff572-0f09-4cd3-bb92-deda90c070a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:37:33.245845 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1352/2698] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jun 29 11:37:32 2022] DELETE /identity/v3/projects/7d0081550eb44279a4275e57d92f65e3 => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:37:42.758085 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=4 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:37:43.239781 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=77,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:38:07.482855 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33815d38-64b2-4911-9279-ca3843b67aa2 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:38:07.484612 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-33815d38-64b2-4911-9279-ca3843b67aa2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:38:07.497737 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33815d38-64b2-4911-9279-ca3843b67aa2 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:38:07.552749 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb75fe4f-38f2-46f1-b130-ebbc9d6a69f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:38:07.553610 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb75fe4f-38f2-46f1-b130-ebbc9d6a69f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:38:07.553893 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb75fe4f-38f2-46f1-b130-ebbc9d6a69f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:38:07.554168 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb75fe4f-38f2-46f1-b130-ebbc9d6a69f0 admin admin] PATH_INFO: `/v3/users/784378f7f2714fe0969344d26b871a9d` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:38:07.555346 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb75fe4f-38f2-46f1-b130-ebbc9d6a69f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=784378f7f2714fe0969344d26b871a9d)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:38:07.557325 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb75fe4f-38f2-46f1-b130-ebbc9d6a69f0 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:38:07.739990 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-eb75fe4f-38f2-46f1-b130-ebbc9d6a69f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '784378f7f2714fe0969344d26b871a9d'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:38:07.813560 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-eb75fe4f-38f2-46f1-b130-ebbc9d6a69f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '784378f7f2714fe0969344d26b871a9d'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:38:07.814183 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-eb75fe4f-38f2-46f1-b130-ebbc9d6a69f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '784378f7f2714fe0969344d26b871a9d'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:38:07.826681 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-eb75fe4f-38f2-46f1-b130-ebbc9d6a69f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '784378f7f2714fe0969344d26b871a9d'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:38:07.827519 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-eb75fe4f-38f2-46f1-b130-ebbc9d6a69f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '784378f7f2714fe0969344d26b871a9d'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:38:07.840211 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1347/2699] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jun 29 11:38:07 2022] DELETE /identity/v3/users/784378f7f2714fe0969344d26b871a9d => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:38:07.844741 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4a3247-d1b9-4864-8969-0658cafc02ab None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:38:07.846047 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4a3247-d1b9-4864-8969-0658cafc02ab None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:38:07.846047 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4a3247-d1b9-4864-8969-0658cafc02ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:38:07.863545 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-2f4a3247-d1b9-4864-8969-0658cafc02ab None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:38:07.864793 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-2f4a3247-d1b9-4864-8969-0658cafc02ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:38:07.927190 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1353/2700] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:38:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:38:07.937535 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb75fe4f-38f2-46f1-b130-ebbc9d6a69f0 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:38:07.938986 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-eb75fe4f-38f2-46f1-b130-ebbc9d6a69f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:38:07.994645 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb75fe4f-38f2-46f1-b130-ebbc9d6a69f0 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:38:07.998748 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b03a8f-e9a5-413a-bd27-98511b15fffb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f4bb9f989e4024a5da28dced646b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:38:07.999386 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b03a8f-e9a5-413a-bd27-98511b15fffb service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:38:07.999669 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b03a8f-e9a5-413a-bd27-98511b15fffb service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:38:07.999981 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b03a8f-e9a5-413a-bd27-98511b15fffb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:38:08.008930 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b03a8f-e9a5-413a-bd27-98511b15fffb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:38:08.010349 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b03a8f-e9a5-413a-bd27-98511b15fffb service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:38:08.019798 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1348/2701] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:38:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:38:08.415019 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4a3247-d1b9-4864-8969-0658cafc02ab None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:38:08.416489 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-2f4a3247-d1b9-4864-8969-0658cafc02ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:38:08.429296 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4a3247-d1b9-4864-8969-0658cafc02ab None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:38:08.434276 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd83016-133a-4c58-a7a6-be551fdfa9e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:38:08.435140 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd83016-133a-4c58-a7a6-be551fdfa9e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:38:08.435421 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd83016-133a-4c58-a7a6-be551fdfa9e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:38:08.435695 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd83016-133a-4c58-a7a6-be551fdfa9e4 admin admin] PATH_INFO: `/v3/projects/d7c0e0b9977c4b29bb87200870d8357d` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:38:08.436947 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd83016-133a-4c58-a7a6-be551fdfa9e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d7c0e0b9977c4b29bb87200870d8357d)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:38:08.438843 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd83016-133a-4c58-a7a6-be551fdfa9e4 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:38:08.523198 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-5dd83016-133a-4c58-a7a6-be551fdfa9e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd7c0e0b9977c4b29bb87200870d8357d', 'request_id': 'req-5dd83016-133a-4c58-a7a6-be551fdfa9e4'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:38:08.529938 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-5dd83016-133a-4c58-a7a6-be551fdfa9e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd7c0e0b9977c4b29bb87200870d8357d', 'request_id': 'req-5dd83016-133a-4c58-a7a6-be551fdfa9e4'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:38:08.535424 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-5dd83016-133a-4c58-a7a6-be551fdfa9e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd7c0e0b9977c4b29bb87200870d8357d', 'request_id': 'req-5dd83016-133a-4c58-a7a6-be551fdfa9e4'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:38:08.584915 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-5dd83016-133a-4c58-a7a6-be551fdfa9e4 admin admin] The token cache is being invalidate because project d7c0e0b9977c4b29bb87200870d8357d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 11:38:08.585209 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-5dd83016-133a-4c58-a7a6-be551fdfa9e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:38:08.585849 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-5dd83016-133a-4c58-a7a6-be551fdfa9e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:38:08.587396 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1354/2702] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jun 29 11:38:08 2022] DELETE /identity/v3/projects/d7c0e0b9977c4b29bb87200870d8357d => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:38:08.593009 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a573cdd5-8e7c-4967-9536-2d065e7f5028 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:38:08.593942 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a573cdd5-8e7c-4967-9536-2d065e7f5028 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:38:08.593942 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a573cdd5-8e7c-4967-9536-2d065e7f5028 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:38:08.611033 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-a573cdd5-8e7c-4967-9536-2d065e7f5028 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:38:08.612280 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-a573cdd5-8e7c-4967-9536-2d065e7f5028 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:38:08.629023 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:38:08.671063 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1349/2703] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:38:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:38:08.678532 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69fa2d4-b579-450b-a71d-ab8b2275055d None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:38:08.678821 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69fa2d4-b579-450b-a71d-ab8b2275055d None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:38:08.679126 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69fa2d4-b579-450b-a71d-ab8b2275055d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:38:08.697134 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-e69fa2d4-b579-450b-a71d-ab8b2275055d None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:38:08.698394 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-e69fa2d4-b579-450b-a71d-ab8b2275055d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:38:08.707112 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1355/2704] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:38:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:38:08.711172 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a573cdd5-8e7c-4967-9536-2d065e7f5028 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:38:08.721291 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a573cdd5-8e7c-4967-9536-2d065e7f5028 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:38:08.725844 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eede8ca-b6bf-450f-b5c4-7340d8128694 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:38:08.726494 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eede8ca-b6bf-450f-b5c4-7340d8128694 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:38:08.726779 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eede8ca-b6bf-450f-b5c4-7340d8128694 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:38:08.727059 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eede8ca-b6bf-450f-b5c4-7340d8128694 admin admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:38:08.727782 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eede8ca-b6bf-450f-b5c4-7340d8128694 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:38:08.728123 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eede8ca-b6bf-450f-b5c4-7340d8128694 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:38:08.729718 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eede8ca-b6bf-450f-b5c4-7340d8128694 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:38:08.742078 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1350/2705] 10.222.0.44 () {58 vars in 1152 bytes} [Wed Jun 29 11:38:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:38:08.746102 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69fa2d4-b579-450b-a71d-ab8b2275055d None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:38:08.755213 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69fa2d4-b579-450b-a71d-ab8b2275055d None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:38:08.759873 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e664014-cf14-4fe8-8ac3-56aedba2fe91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:38:08.760524 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e664014-cf14-4fe8-8ac3-56aedba2fe91 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:38:08.760808 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e664014-cf14-4fe8-8ac3-56aedba2fe91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:38:08.761084 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e664014-cf14-4fe8-8ac3-56aedba2fe91 admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:38:08.762283 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e664014-cf14-4fe8-8ac3-56aedba2fe91 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:38:08.764194 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e664014-cf14-4fe8-8ac3-56aedba2fe91 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:38:08.879583 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1356/2706] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jun 29 11:38:08 2022] POST /identity/v3/projects => generated 373 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 11:38:08.883639 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eede8ca-b6bf-450f-b5c4-7340d8128694 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:38:08.894468 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eede8ca-b6bf-450f-b5c4-7340d8128694 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:38:08.952008 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aae4687-458a-4da1-a930-72348da1c0ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:38:08.952685 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aae4687-458a-4da1-a930-72348da1c0ff admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:38:08.952967 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aae4687-458a-4da1-a930-72348da1c0ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:38:08.953245 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aae4687-458a-4da1-a930-72348da1c0ff admin admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:38:08.953943 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aae4687-458a-4da1-a930-72348da1c0ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:38:08.955886 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aae4687-458a-4da1-a930-72348da1c0ff admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:38:09.038097 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1351/2707] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jun 29 11:38:08 2022] POST /identity/v3/users => generated 287 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:38:09.042295 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e664014-cf14-4fe8-8ac3-56aedba2fe91 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:38:09.052447 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e664014-cf14-4fe8-8ac3-56aedba2fe91 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:38:09.057107 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dda0aac-9424-4d36-b368-d261ede58b4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:38:09.057749 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dda0aac-9424-4d36-b368-d261ede58b4c admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:38:09.058031 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dda0aac-9424-4d36-b368-d261ede58b4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:38:09.058306 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dda0aac-9424-4d36-b368-d261ede58b4c admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:38:09.059016 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dda0aac-9424-4d36-b368-d261ede58b4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:38:09.061143 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dda0aac-9424-4d36-b368-d261ede58b4c admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:38:09.071280 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1357/2708] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:38:09 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 11:38:09.075683 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aae4687-458a-4da1-a930-72348da1c0ff admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:38:09.085497 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aae4687-458a-4da1-a930-72348da1c0ff admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:38:09.090075 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c902697-85a7-4906-a79c-0401434ee755 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:38:09.090716 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c902697-85a7-4906-a79c-0401434ee755 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:38:09.091002 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c902697-85a7-4906-a79c-0401434ee755 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:38:09.091285 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c902697-85a7-4906-a79c-0401434ee755 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:38:09.092036 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c902697-85a7-4906-a79c-0401434ee755 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:38:09.093616 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c902697-85a7-4906-a79c-0401434ee755 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:38:09.103216 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1352/2709] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:38:09 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 11:38:09.106987 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dda0aac-9424-4d36-b368-d261ede58b4c admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:38:09.116818 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dda0aac-9424-4d36-b368-d261ede58b4c admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:38:09.121521 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddf5598-83ec-413d-bb6f-be38e7bb86e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:38:09.122894 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddf5598-83ec-413d-bb6f-be38e7bb86e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:38:09.123203 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddf5598-83ec-413d-bb6f-be38e7bb86e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:38:09.123476 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddf5598-83ec-413d-bb6f-be38e7bb86e0 admin admin] PATH_INFO: `/v3/projects/c62a549e8c494cdabaf1ddef15a5d328/users/46ba1fd6613940f5b2d47f47c04ae60f/roles/e1c88eebe911432cab6a07fbeb84653b` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:38:09.138643 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddf5598-83ec-413d-bb6f-be38e7bb86e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c62a549e8c494cdabaf1ddef15a5d328, user_id=46ba1fd6613940f5b2d47f47c04ae60f, role_id=e1c88eebe911432cab6a07fbeb84653b)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:38:09.141233 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddf5598-83ec-413d-bb6f-be38e7bb86e0 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:38:09.214723 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1358/2710] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jun 29 11:38:09 2022] PUT /identity/v3/projects/c62a549e8c494cdabaf1ddef15a5d328/users/46ba1fd6613940f5b2d47f47c04ae60f/roles/e1c88eebe911432cab6a07fbeb84653b => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:38:09.225213 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcfb700-a175-4ba0-bb69-73eb9cc1742d None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:38:09.225506 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcfb700-a175-4ba0-bb69-73eb9cc1742d None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:38:09.225784 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcfb700-a175-4ba0-bb69-73eb9cc1742d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:38:09.252930 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-bbcfb700-a175-4ba0-bb69-73eb9cc1742d None None] MFA Rules not processed for user `46ba1fd6613940f5b2d47f47c04ae60f`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:38:09.254138 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-bbcfb700-a175-4ba0-bb69-73eb9cc1742d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:38:09.315847 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1353/2711] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:38:09 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:38:09.326274 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddf5598-83ec-413d-bb6f-be38e7bb86e0 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:38:09.327960 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-2ddf5598-83ec-413d-bb6f-be38e7bb86e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:38:09.349052 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:38:09.383674 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddf5598-83ec-413d-bb6f-be38e7bb86e0 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:38:09.387754 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11fdcff-e1d5-4d3f-a954-7beef2db3857 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff7931b75a924394b1af733febce0834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:38:09.388417 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11fdcff-e1d5-4d3f-a954-7beef2db3857 service cinder] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:38:09.388755 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11fdcff-e1d5-4d3f-a954-7beef2db3857 service cinder] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:38:09.388985 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11fdcff-e1d5-4d3f-a954-7beef2db3857 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:38:09.397057 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11fdcff-e1d5-4d3f-a954-7beef2db3857 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:38:09.398623 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11fdcff-e1d5-4d3f-a954-7beef2db3857 service cinder] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:38:09.407238 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1359/2712] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:38:09 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:38:19.313083 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=89,keystone:INSERT=3 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:38:19.443202 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=17 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:38:19.800195 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcfb700-a175-4ba0-bb69-73eb9cc1742d None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:38:19.801681 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-bbcfb700-a175-4ba0-bb69-73eb9cc1742d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:38:19.900892 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcfb700-a175-4ba0-bb69-73eb9cc1742d None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:38:19.906426 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417274a3-fb91-42fc-8929-fc4825522711 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '796aeeff16df4739bddec1f6dee61bfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:38:19.907066 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417274a3-fb91-42fc-8929-fc4825522711 service nova] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:38:19.907355 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417274a3-fb91-42fc-8929-fc4825522711 service nova] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:38:19.907633 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417274a3-fb91-42fc-8929-fc4825522711 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:38:19.909420 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-417274a3-fb91-42fc-8929-fc4825522711 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:38:19.966699 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417274a3-fb91-42fc-8929-fc4825522711 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:38:19.968350 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417274a3-fb91-42fc-8929-fc4825522711 service nova] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:38:19.977252 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1354/2713] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:38:19 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:38:21.332339 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11fdcff-e1d5-4d3f-a954-7beef2db3857 service cinder] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:38:21.394660 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11fdcff-e1d5-4d3f-a954-7beef2db3857 service cinder] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:38:21.399268 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660bfb01-13b5-4271-af13-0ecd79cc8047 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:38:21.400186 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660bfb01-13b5-4271-af13-0ecd79cc8047 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:38:21.401429 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660bfb01-13b5-4271-af13-0ecd79cc8047 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:38:21.401429 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660bfb01-13b5-4271-af13-0ecd79cc8047 admin admin] PATH_INFO: `/v3/users/46ba1fd6613940f5b2d47f47c04ae60f` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:38:21.401952 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660bfb01-13b5-4271-af13-0ecd79cc8047 admin admin] RBAC: Authorizing `identity:delete_user(user_id=46ba1fd6613940f5b2d47f47c04ae60f)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:38:21.403879 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660bfb01-13b5-4271-af13-0ecd79cc8047 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:38:21.618492 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-660bfb01-13b5-4271-af13-0ecd79cc8047 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '46ba1fd6613940f5b2d47f47c04ae60f'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:38:21.619178 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-660bfb01-13b5-4271-af13-0ecd79cc8047 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '46ba1fd6613940f5b2d47f47c04ae60f'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:38:21.700370 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-660bfb01-13b5-4271-af13-0ecd79cc8047 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '46ba1fd6613940f5b2d47f47c04ae60f'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:38:21.708096 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-660bfb01-13b5-4271-af13-0ecd79cc8047 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '46ba1fd6613940f5b2d47f47c04ae60f'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:38:21.710367 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-660bfb01-13b5-4271-af13-0ecd79cc8047 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '46ba1fd6613940f5b2d47f47c04ae60f'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:38:21.727166 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1360/2714] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jun 29 11:38:21 2022] DELETE /identity/v3/users/46ba1fd6613940f5b2d47f47c04ae60f => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:38:21.731676 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3ebf17-9981-4d91-be7b-3b1053bbdabd None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:38:21.732187 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3ebf17-9981-4d91-be7b-3b1053bbdabd None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:38:21.732291 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3ebf17-9981-4d91-be7b-3b1053bbdabd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:38:21.749548 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-5c3ebf17-9981-4d91-be7b-3b1053bbdabd None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:38:21.750753 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-5c3ebf17-9981-4d91-be7b-3b1053bbdabd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:38:21.821405 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1355/2715] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:38:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:38:21.831065 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660bfb01-13b5-4271-af13-0ecd79cc8047 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:38:21.832542 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-660bfb01-13b5-4271-af13-0ecd79cc8047 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:38:21.889899 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660bfb01-13b5-4271-af13-0ecd79cc8047 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:38:21.894341 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe13f7c8-499f-4b03-beb9-c0d932545733 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f4bb9f989e4024a5da28dced646b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:38:21.894998 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe13f7c8-499f-4b03-beb9-c0d932545733 service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:38:21.895285 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe13f7c8-499f-4b03-beb9-c0d932545733 service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:38:21.895565 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe13f7c8-499f-4b03-beb9-c0d932545733 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:38:21.905209 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe13f7c8-499f-4b03-beb9-c0d932545733 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:38:21.907228 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe13f7c8-499f-4b03-beb9-c0d932545733 service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:38:21.917955 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1361/2716] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:38:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:38:22.330758 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3ebf17-9981-4d91-be7b-3b1053bbdabd None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:38:22.336882 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-5c3ebf17-9981-4d91-be7b-3b1053bbdabd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:38:22.344502 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3ebf17-9981-4d91-be7b-3b1053bbdabd None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:38:22.349505 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580cd5aa-c6b6-4c93-a1d2-93dd0727b80c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:38:22.350367 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580cd5aa-c6b6-4c93-a1d2-93dd0727b80c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:38:22.350650 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580cd5aa-c6b6-4c93-a1d2-93dd0727b80c admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:38:22.350925 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580cd5aa-c6b6-4c93-a1d2-93dd0727b80c admin admin] PATH_INFO: `/v3/projects/c62a549e8c494cdabaf1ddef15a5d328` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:38:22.352188 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580cd5aa-c6b6-4c93-a1d2-93dd0727b80c admin admin] RBAC: Authorizing `identity:delete_project(project_id=c62a549e8c494cdabaf1ddef15a5d328)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:38:22.354097 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580cd5aa-c6b6-4c93-a1d2-93dd0727b80c admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:38:22.365990 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:38:22.490206 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-580cd5aa-c6b6-4c93-a1d2-93dd0727b80c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c62a549e8c494cdabaf1ddef15a5d328', 'request_id': 'req-580cd5aa-c6b6-4c93-a1d2-93dd0727b80c'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:38:22.550693 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-580cd5aa-c6b6-4c93-a1d2-93dd0727b80c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c62a549e8c494cdabaf1ddef15a5d328', 'request_id': 'req-580cd5aa-c6b6-4c93-a1d2-93dd0727b80c'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:38:22.557534 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-580cd5aa-c6b6-4c93-a1d2-93dd0727b80c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c62a549e8c494cdabaf1ddef15a5d328', 'request_id': 'req-580cd5aa-c6b6-4c93-a1d2-93dd0727b80c'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:38:22.569708 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-580cd5aa-c6b6-4c93-a1d2-93dd0727b80c admin admin] The token cache is being invalidate because project c62a549e8c494cdabaf1ddef15a5d328 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 11:38:22.570000 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-580cd5aa-c6b6-4c93-a1d2-93dd0727b80c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:38:22.570637 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-580cd5aa-c6b6-4c93-a1d2-93dd0727b80c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:38:22.572374 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1356/2717] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jun 29 11:38:22 2022] DELETE /identity/v3/projects/c62a549e8c494cdabaf1ddef15a5d328 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 11:38:22.578554 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b07852-12b6-44a5-ab16-9c0d3065e916 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:38:22.579036 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b07852-12b6-44a5-ab16-9c0d3065e916 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:38:22.579144 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b07852-12b6-44a5-ab16-9c0d3065e916 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:38:22.596456 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-21b07852-12b6-44a5-ab16-9c0d3065e916 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:38:22.597760 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-21b07852-12b6-44a5-ab16-9c0d3065e916 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:38:22.614867 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:38:22.659501 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1362/2718] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:38:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:38:22.718594 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b400894-5fa1-4d53-978e-a183f197fe29 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:38:22.718952 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b400894-5fa1-4d53-978e-a183f197fe29 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:38:22.719282 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b400894-5fa1-4d53-978e-a183f197fe29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:38:22.738442 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-3b400894-5fa1-4d53-978e-a183f197fe29 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:38:22.743968 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-3b400894-5fa1-4d53-978e-a183f197fe29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:38:22.747762 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1357/2719] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:38:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:38:22.752784 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b07852-12b6-44a5-ab16-9c0d3065e916 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:38:22.762571 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b07852-12b6-44a5-ab16-9c0d3065e916 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:38:22.767729 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa50439-e69c-47db-b36e-1b4a58e39c2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:38:22.769292 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa50439-e69c-47db-b36e-1b4a58e39c2b admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:38:22.769292 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa50439-e69c-47db-b36e-1b4a58e39c2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:38:22.769292 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa50439-e69c-47db-b36e-1b4a58e39c2b admin admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:38:22.769732 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa50439-e69c-47db-b36e-1b4a58e39c2b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:38:22.770050 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa50439-e69c-47db-b36e-1b4a58e39c2b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:38:22.771631 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa50439-e69c-47db-b36e-1b4a58e39c2b admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:38:22.783293 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1363/2720] 10.222.0.44 () {58 vars in 1152 bytes} [Wed Jun 29 11:38:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:38:22.787558 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b400894-5fa1-4d53-978e-a183f197fe29 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:38:22.797521 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b400894-5fa1-4d53-978e-a183f197fe29 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:38:22.805206 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53506a8-01c2-4187-97ea-0d769bae144b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:38:22.805206 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53506a8-01c2-4187-97ea-0d769bae144b admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:38:22.805206 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53506a8-01c2-4187-97ea-0d769bae144b admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:38:22.805206 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53506a8-01c2-4187-97ea-0d769bae144b admin admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:38:22.805620 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53506a8-01c2-4187-97ea-0d769bae144b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:38:22.805880 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53506a8-01c2-4187-97ea-0d769bae144b admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:38:22.953370 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1358/2721] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jun 29 11:38:22 2022] POST /identity/v3/projects => generated 381 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:38:22.957734 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa50439-e69c-47db-b36e-1b4a58e39c2b admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:38:22.967601 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa50439-e69c-47db-b36e-1b4a58e39c2b admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:38:23.024253 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c0bfab-92a6-4c06-b431-d610f96d3c3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:38:23.024953 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c0bfab-92a6-4c06-b431-d610f96d3c3e admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:38:23.025240 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c0bfab-92a6-4c06-b431-d610f96d3c3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:38:23.025518 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c0bfab-92a6-4c06-b431-d610f96d3c3e admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:38:23.026204 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c0bfab-92a6-4c06-b431-d610f96d3c3e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:38:23.028145 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c0bfab-92a6-4c06-b431-d610f96d3c3e admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:38:23.094863 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1364/2722] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jun 29 11:38:22 2022] POST /identity/v3/users => generated 291 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:38:23.099086 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53506a8-01c2-4187-97ea-0d769bae144b admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:38:23.110081 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53506a8-01c2-4187-97ea-0d769bae144b admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:38:23.114656 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb9012e-6a20-4289-be7a-3556bdb2a705 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:38:23.115382 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb9012e-6a20-4289-be7a-3556bdb2a705 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:38:23.115667 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb9012e-6a20-4289-be7a-3556bdb2a705 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:38:23.115981 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb9012e-6a20-4289-be7a-3556bdb2a705 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:38:23.116787 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb9012e-6a20-4289-be7a-3556bdb2a705 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:38:23.118429 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb9012e-6a20-4289-be7a-3556bdb2a705 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:38:23.128787 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1359/2723] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:38:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 11:38:23.132895 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c0bfab-92a6-4c06-b431-d610f96d3c3e admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:38:23.143347 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c0bfab-92a6-4c06-b431-d610f96d3c3e admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:38:23.148093 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda1a03f-ded9-4fea-b7ee-fc2b28d5d11d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:38:23.148778 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda1a03f-ded9-4fea-b7ee-fc2b28d5d11d admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:38:23.149064 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda1a03f-ded9-4fea-b7ee-fc2b28d5d11d admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:38:23.149374 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda1a03f-ded9-4fea-b7ee-fc2b28d5d11d admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:38:23.150094 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda1a03f-ded9-4fea-b7ee-fc2b28d5d11d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:38:23.151774 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda1a03f-ded9-4fea-b7ee-fc2b28d5d11d admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:38:23.162298 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1365/2724] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:38:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 11:38:23.166366 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb9012e-6a20-4289-be7a-3556bdb2a705 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:38:23.177100 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb9012e-6a20-4289-be7a-3556bdb2a705 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:38:23.181845 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3063ec5-03da-4c3c-8430-758e500638c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:38:23.182711 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3063ec5-03da-4c3c-8430-758e500638c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:38:23.182995 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3063ec5-03da-4c3c-8430-758e500638c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:38:23.183272 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3063ec5-03da-4c3c-8430-758e500638c9 admin admin] PATH_INFO: `/v3/projects/db642fb7d52f4fdfa95cd85555c79af3/users/8c0159323d8d46ffb28b8071fcf0acf2/roles/e1c88eebe911432cab6a07fbeb84653b` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:38:23.197828 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3063ec5-03da-4c3c-8430-758e500638c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=db642fb7d52f4fdfa95cd85555c79af3, user_id=8c0159323d8d46ffb28b8071fcf0acf2, role_id=e1c88eebe911432cab6a07fbeb84653b)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:38:23.200503 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3063ec5-03da-4c3c-8430-758e500638c9 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:38:23.248908 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1360/2725] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jun 29 11:38:23 2022] PUT /identity/v3/projects/db642fb7d52f4fdfa95cd85555c79af3/users/8c0159323d8d46ffb28b8071fcf0acf2/roles/e1c88eebe911432cab6a07fbeb84653b => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:38:23.254162 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd0b6b7-c20d-440c-a0c5-8e3823503c38 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:38:23.254451 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd0b6b7-c20d-440c-a0c5-8e3823503c38 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:38:23.254725 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd0b6b7-c20d-440c-a0c5-8e3823503c38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:38:23.273060 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-ebd0b6b7-c20d-440c-a0c5-8e3823503c38 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:38:23.277071 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-ebd0b6b7-c20d-440c-a0c5-8e3823503c38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:38:23.333123 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1366/2726] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:38:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:38:23.342769 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3063ec5-03da-4c3c-8430-758e500638c9 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:38:23.344982 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-b3063ec5-03da-4c3c-8430-758e500638c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:38:23.401191 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3063ec5-03da-4c3c-8430-758e500638c9 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:38:23.404937 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98f8cc5-905f-4dfa-ac26-e642f026eab7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f4bb9f989e4024a5da28dced646b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:38:23.405569 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98f8cc5-905f-4dfa-ac26-e642f026eab7 service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:38:23.405851 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98f8cc5-905f-4dfa-ac26-e642f026eab7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:38:23.406124 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98f8cc5-905f-4dfa-ac26-e642f026eab7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:38:23.415354 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98f8cc5-905f-4dfa-ac26-e642f026eab7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:38:23.416810 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98f8cc5-905f-4dfa-ac26-e642f026eab7 service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:38:23.426693 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1361/2727] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:38:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:38:24.577337 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f0333a-334d-422e-9e49-b87602696fa2 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:38:24.577922 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f0333a-334d-422e-9e49-b87602696fa2 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:38:24.578040 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f0333a-334d-422e-9e49-b87602696fa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:38:24.608274 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-70f0333a-334d-422e-9e49-b87602696fa2 None None] MFA Rules not processed for user `8c0159323d8d46ffb28b8071fcf0acf2`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:38:24.609340 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-70f0333a-334d-422e-9e49-b87602696fa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:38:24.630942 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:38:24.681027 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1367/2728] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:38:24 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:38:24.692015 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98f8cc5-905f-4dfa-ac26-e642f026eab7 service neutron] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:38:24.693422 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-d98f8cc5-905f-4dfa-ac26-e642f026eab7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:38:24.730053 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:38:24.752114 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98f8cc5-905f-4dfa-ac26-e642f026eab7 service neutron] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:38:24.758405 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39ddaff-b07f-4956-8e43-54348f2d40d7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff7931b75a924394b1af733febce0834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:38:24.758405 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39ddaff-b07f-4956-8e43-54348f2d40d7 service cinder] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:38:24.758405 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39ddaff-b07f-4956-8e43-54348f2d40d7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:38:24.758405 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39ddaff-b07f-4956-8e43-54348f2d40d7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:38:24.765163 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39ddaff-b07f-4956-8e43-54348f2d40d7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:38:24.766633 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39ddaff-b07f-4956-8e43-54348f2d40d7 service cinder] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:38:24.776042 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1362/2729] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:38:24 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:38:26.427331 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc5172d-5152-44a4-bf0c-989ece12e7cf None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:38:26.427877 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc5172d-5152-44a4-bf0c-989ece12e7cf None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:38:26.427996 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc5172d-5152-44a4-bf0c-989ece12e7cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:38:26.445521 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-0fc5172d-5152-44a4-bf0c-989ece12e7cf None None] MFA Rules not processed for user `8c0159323d8d46ffb28b8071fcf0acf2`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:38:26.447010 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-0fc5172d-5152-44a4-bf0c-989ece12e7cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:38:26.454540 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1368/2730] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:38:26 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:38:26.686258 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39ddaff-b07f-4956-8e43-54348f2d40d7 service cinder] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:38:26.696027 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39ddaff-b07f-4956-8e43-54348f2d40d7 service cinder] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:38:26.700002 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6c4d11-c948-44ee-a711-738f7cfea5b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f4bb9f989e4024a5da28dced646b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:38:26.700632 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6c4d11-c948-44ee-a711-738f7cfea5b8 service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:38:26.700915 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6c4d11-c948-44ee-a711-738f7cfea5b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:38:26.701191 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6c4d11-c948-44ee-a711-738f7cfea5b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:38:26.703036 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-ee6c4d11-c948-44ee-a711-738f7cfea5b8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:38:26.713180 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6c4d11-c948-44ee-a711-738f7cfea5b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:38:26.714595 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6c4d11-c948-44ee-a711-738f7cfea5b8 service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:38:26.722420 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1363/2731] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:38:26 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:38:36.441085 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=22 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:38:36.719176 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=17 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:38:42.004436 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01444ae5-e7f7-4e90-9cbd-b23797b0c9dc None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:38:42.004937 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01444ae5-e7f7-4e90-9cbd-b23797b0c9dc None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:38:42.004937 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01444ae5-e7f7-4e90-9cbd-b23797b0c9dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:38:42.022072 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-01444ae5-e7f7-4e90-9cbd-b23797b0c9dc None None] MFA Rules not processed for user `cf7bc4681c954c158a7051046bd9aa47`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:38:42.029517 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-01444ae5-e7f7-4e90-9cbd-b23797b0c9dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:38:42.078022 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1369/2732] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:38:42 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:38:43.928527 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6c4d11-c948-44ee-a711-738f7cfea5b8 service neutron] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:38:43.930562 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-ee6c4d11-c948-44ee-a711-738f7cfea5b8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:38:44.028724 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6c4d11-c948-44ee-a711-738f7cfea5b8 service neutron] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:38:44.034596 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179b31a0-6cb2-46f1-9bc2-70bd3c4cc59c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a0505342b704d02977a5125957c2c9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:38:44.035237 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179b31a0-6cb2-46f1-9bc2-70bd3c4cc59c service placement] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:38:44.035557 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179b31a0-6cb2-46f1-9bc2-70bd3c4cc59c service placement] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:38:44.035838 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179b31a0-6cb2-46f1-9bc2-70bd3c4cc59c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:38:44.037620 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-179b31a0-6cb2-46f1-9bc2-70bd3c4cc59c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:38:44.048906 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179b31a0-6cb2-46f1-9bc2-70bd3c4cc59c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:38:44.050489 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179b31a0-6cb2-46f1-9bc2-70bd3c4cc59c service placement] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:38:44.060878 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1364/2733] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:38:43 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:38:52.070985 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=25 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:38:54.056101 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=41 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:39:49.211726 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7939cdb7-b89c-4d33-aae7-7b27579aa7e8 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:39:49.212260 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7939cdb7-b89c-4d33-aae7-7b27579aa7e8 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:39:49.212364 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7939cdb7-b89c-4d33-aae7-7b27579aa7e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:39:49.229547 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-7939cdb7-b89c-4d33-aae7-7b27579aa7e8 None None] MFA Rules not processed for user `8c0159323d8d46ffb28b8071fcf0acf2`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:39:49.230735 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-7939cdb7-b89c-4d33-aae7-7b27579aa7e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:39:49.237770 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1370/2734] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:39:49 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:39:58.124727 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179b31a0-6cb2-46f1-9bc2-70bd3c4cc59c service placement] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:39:58.134933 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179b31a0-6cb2-46f1-9bc2-70bd3c4cc59c service placement] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:39:58.139119 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a606d7d-2b22-49ee-8165-f97c7c69bdb2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f4bb9f989e4024a5da28dced646b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:39:58.139747 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a606d7d-2b22-49ee-8165-f97c7c69bdb2 service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:39:58.140060 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a606d7d-2b22-49ee-8165-f97c7c69bdb2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:39:58.140336 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a606d7d-2b22-49ee-8165-f97c7c69bdb2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:39:58.142088 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-5a606d7d-2b22-49ee-8165-f97c7c69bdb2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:39:58.218052 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a606d7d-2b22-49ee-8165-f97c7c69bdb2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:39:58.219487 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a606d7d-2b22-49ee-8165-f97c7c69bdb2 service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:39:58.230233 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1365/2735] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:39:58 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:39:59.224376 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=3 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:40:08.225388 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=34 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:40:30.794558 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7939cdb7-b89c-4d33-aae7-7b27579aa7e8 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:40:30.805046 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7939cdb7-b89c-4d33-aae7-7b27579aa7e8 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:40:30.808706 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e1fae1-008a-47a0-879a-3fcc7064ffba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f4bb9f989e4024a5da28dced646b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:40:30.809348 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e1fae1-008a-47a0-879a-3fcc7064ffba service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:40:30.809634 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e1fae1-008a-47a0-879a-3fcc7064ffba service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:40:30.809910 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e1fae1-008a-47a0-879a-3fcc7064ffba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:40:30.811771 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-68e1fae1-008a-47a0-879a-3fcc7064ffba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:40:30.825922 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e1fae1-008a-47a0-879a-3fcc7064ffba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:40:30.827332 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e1fae1-008a-47a0-879a-3fcc7064ffba service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:40:30.838529 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1371/2736] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:40:30 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:40:40.833509 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=6 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:40:55.673114 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a606d7d-2b22-49ee-8165-f97c7c69bdb2 service neutron] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:40:55.682069 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a606d7d-2b22-49ee-8165-f97c7c69bdb2 service neutron] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:40:55.686901 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ecd4e4-a099-40f1-9ca0-51dbc8f791c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f4bb9f989e4024a5da28dced646b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:40:55.687556 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ecd4e4-a099-40f1-9ca0-51dbc8f791c1 service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:40:55.687863 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ecd4e4-a099-40f1-9ca0-51dbc8f791c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:40:55.688146 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ecd4e4-a099-40f1-9ca0-51dbc8f791c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:40:55.690545 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-55ecd4e4-a099-40f1-9ca0-51dbc8f791c1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:40:55.701022 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ecd4e4-a099-40f1-9ca0-51dbc8f791c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:40:55.702500 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ecd4e4-a099-40f1-9ca0-51dbc8f791c1 service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:40:55.711530 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1366/2737] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:40:55 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:41:05.453293 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e1fae1-008a-47a0-879a-3fcc7064ffba service neutron] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:41:05.465305 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e1fae1-008a-47a0-879a-3fcc7064ffba service neutron] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:41:05.470218 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457eee5e-f094-4d80-b69d-7bd1f93f4e36 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a0505342b704d02977a5125957c2c9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:41:05.470975 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457eee5e-f094-4d80-b69d-7bd1f93f4e36 service placement] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:41:05.471288 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457eee5e-f094-4d80-b69d-7bd1f93f4e36 service placement] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:41:05.471565 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457eee5e-f094-4d80-b69d-7bd1f93f4e36 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:41:05.473978 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-457eee5e-f094-4d80-b69d-7bd1f93f4e36 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:41:05.489172 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457eee5e-f094-4d80-b69d-7bd1f93f4e36 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:41:05.493411 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457eee5e-f094-4d80-b69d-7bd1f93f4e36 service placement] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:41:05.504495 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1372/2738] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:41:05 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:41:05.556228 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ecd4e4-a099-40f1-9ca0-51dbc8f791c1 service neutron] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:41:05.568314 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ecd4e4-a099-40f1-9ca0-51dbc8f791c1 service neutron] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:41:05.573382 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5f8b55-8b3c-46ed-8812-1b53a4c24609 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:41:05.574333 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5f8b55-8b3c-46ed-8812-1b53a4c24609 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:41:05.574615 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5f8b55-8b3c-46ed-8812-1b53a4c24609 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:41:05.575190 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5f8b55-8b3c-46ed-8812-1b53a4c24609 admin admin] PATH_INFO: `/v3/users/8c0159323d8d46ffb28b8071fcf0acf2` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:41:05.576157 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5f8b55-8b3c-46ed-8812-1b53a4c24609 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c0159323d8d46ffb28b8071fcf0acf2)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:41:05.578212 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5f8b55-8b3c-46ed-8812-1b53a4c24609 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:41:05.763167 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-9c5f8b55-8b3c-46ed-8812-1b53a4c24609 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c0159323d8d46ffb28b8071fcf0acf2'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:41:05.806117 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-9c5f8b55-8b3c-46ed-8812-1b53a4c24609 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c0159323d8d46ffb28b8071fcf0acf2'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:41:05.806702 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-9c5f8b55-8b3c-46ed-8812-1b53a4c24609 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c0159323d8d46ffb28b8071fcf0acf2'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:41:05.818943 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-9c5f8b55-8b3c-46ed-8812-1b53a4c24609 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c0159323d8d46ffb28b8071fcf0acf2'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:41:05.824504 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-9c5f8b55-8b3c-46ed-8812-1b53a4c24609 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c0159323d8d46ffb28b8071fcf0acf2'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:41:05.832279 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1367/2739] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jun 29 11:41:05 2022] DELETE /identity/v3/users/8c0159323d8d46ffb28b8071fcf0acf2 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:41:06.065038 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457eee5e-f094-4d80-b69d-7bd1f93f4e36 service placement] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:41:06.076247 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457eee5e-f094-4d80-b69d-7bd1f93f4e36 service placement] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:41:06.134806 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c63cf6-799c-4687-af58-0114b7b7e0ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:41:06.135667 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c63cf6-799c-4687-af58-0114b7b7e0ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:41:06.135993 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c63cf6-799c-4687-af58-0114b7b7e0ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:41:06.136266 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c63cf6-799c-4687-af58-0114b7b7e0ab admin admin] PATH_INFO: `/v3/projects/db642fb7d52f4fdfa95cd85555c79af3` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:41:06.137475 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c63cf6-799c-4687-af58-0114b7b7e0ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=db642fb7d52f4fdfa95cd85555c79af3)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:41:06.139359 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c63cf6-799c-4687-af58-0114b7b7e0ab admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:41:06.243398 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-f2c63cf6-799c-4687-af58-0114b7b7e0ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db642fb7d52f4fdfa95cd85555c79af3', 'request_id': 'req-f2c63cf6-799c-4687-af58-0114b7b7e0ab'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:41:06.249703 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-f2c63cf6-799c-4687-af58-0114b7b7e0ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db642fb7d52f4fdfa95cd85555c79af3', 'request_id': 'req-f2c63cf6-799c-4687-af58-0114b7b7e0ab'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:41:06.255789 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-f2c63cf6-799c-4687-af58-0114b7b7e0ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db642fb7d52f4fdfa95cd85555c79af3', 'request_id': 'req-f2c63cf6-799c-4687-af58-0114b7b7e0ab'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:41:06.308636 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-f2c63cf6-799c-4687-af58-0114b7b7e0ab admin admin] The token cache is being invalidate because project db642fb7d52f4fdfa95cd85555c79af3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 11:41:06.308865 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-f2c63cf6-799c-4687-af58-0114b7b7e0ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:41:06.309478 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-f2c63cf6-799c-4687-af58-0114b7b7e0ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:41:06.311129 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1373/2740] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jun 29 11:41:06 2022] DELETE /identity/v3/projects/db642fb7d52f4fdfa95cd85555c79af3 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:41:06.316855 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1800a3-d1a7-4e7c-9a88-a2dc9b20c3b0 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:41:06.317148 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1800a3-d1a7-4e7c-9a88-a2dc9b20c3b0 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:41:06.317429 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1800a3-d1a7-4e7c-9a88-a2dc9b20c3b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:41:06.334720 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-7c1800a3-d1a7-4e7c-9a88-a2dc9b20c3b0 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:41:06.335987 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-7c1800a3-d1a7-4e7c-9a88-a2dc9b20c3b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:41:06.393923 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1368/2741] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:41:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:41:06.402548 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55821e20-e2a9-4b71-b7af-107c4f38f900 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:41:06.402837 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55821e20-e2a9-4b71-b7af-107c4f38f900 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:41:06.403114 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55821e20-e2a9-4b71-b7af-107c4f38f900 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:41:06.421949 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-55821e20-e2a9-4b71-b7af-107c4f38f900 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:41:06.423139 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-55821e20-e2a9-4b71-b7af-107c4f38f900 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:41:06.431572 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1374/2742] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:41:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:41:06.435939 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1800a3-d1a7-4e7c-9a88-a2dc9b20c3b0 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:41:06.445638 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1800a3-d1a7-4e7c-9a88-a2dc9b20c3b0 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:41:06.450207 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c651fb-7993-4fe1-aa79-e50b54ade377 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:41:06.450856 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c651fb-7993-4fe1-aa79-e50b54ade377 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:41:06.451134 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c651fb-7993-4fe1-aa79-e50b54ade377 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:41:06.451418 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c651fb-7993-4fe1-aa79-e50b54ade377 admin admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:41:06.452189 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c651fb-7993-4fe1-aa79-e50b54ade377 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:41:06.452502 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c651fb-7993-4fe1-aa79-e50b54ade377 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:41:06.454097 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c651fb-7993-4fe1-aa79-e50b54ade377 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:41:06.465395 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1369/2743] 10.222.0.44 () {58 vars in 1152 bytes} [Wed Jun 29 11:41:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:41:06.469433 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55821e20-e2a9-4b71-b7af-107c4f38f900 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:41:06.478975 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55821e20-e2a9-4b71-b7af-107c4f38f900 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:41:06.483538 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c1c322-23fd-4c3c-93dc-d5f76cf2533b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:41:06.484209 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c1c322-23fd-4c3c-93dc-d5f76cf2533b admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:41:06.484493 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c1c322-23fd-4c3c-93dc-d5f76cf2533b admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:41:06.484769 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c1c322-23fd-4c3c-93dc-d5f76cf2533b admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:41:06.485445 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c1c322-23fd-4c3c-93dc-d5f76cf2533b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:41:06.487566 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c1c322-23fd-4c3c-93dc-d5f76cf2533b admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:41:06.576526 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1375/2744] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jun 29 11:41:06 2022] POST /identity/v3/projects => generated 373 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:41:06.578270 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c651fb-7993-4fe1-aa79-e50b54ade377 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:41:06.588476 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c651fb-7993-4fe1-aa79-e50b54ade377 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:41:06.631698 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:41:06.644450 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9253628-b7dd-4a07-a59b-460cb0753c88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:41:06.645196 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9253628-b7dd-4a07-a59b-460cb0753c88 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:41:06.645484 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9253628-b7dd-4a07-a59b-460cb0753c88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:41:06.645816 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9253628-b7dd-4a07-a59b-460cb0753c88 admin admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:41:06.646522 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9253628-b7dd-4a07-a59b-460cb0753c88 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:41:06.648506 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9253628-b7dd-4a07-a59b-460cb0753c88 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:41:06.772620 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1370/2745] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jun 29 11:41:06 2022] POST /identity/v3/users => generated 287 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:41:06.776686 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c1c322-23fd-4c3c-93dc-d5f76cf2533b admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:41:06.793226 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c1c322-23fd-4c3c-93dc-d5f76cf2533b admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:41:06.799675 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7370df0a-ce8c-4914-a99d-5f7a69df6a45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:41:06.799675 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7370df0a-ce8c-4914-a99d-5f7a69df6a45 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:41:06.799675 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7370df0a-ce8c-4914-a99d-5f7a69df6a45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:41:06.799675 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7370df0a-ce8c-4914-a99d-5f7a69df6a45 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:41:06.800120 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7370df0a-ce8c-4914-a99d-5f7a69df6a45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:41:06.801671 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7370df0a-ce8c-4914-a99d-5f7a69df6a45 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:41:06.812506 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1376/2746] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:41:06 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 11:41:06.816650 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9253628-b7dd-4a07-a59b-460cb0753c88 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:41:06.826708 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9253628-b7dd-4a07-a59b-460cb0753c88 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:41:06.832208 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2182ea-6053-455c-9f89-14e2ce77040a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:41:06.832887 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2182ea-6053-455c-9f89-14e2ce77040a admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:41:06.835024 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2182ea-6053-455c-9f89-14e2ce77040a admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:41:06.835024 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2182ea-6053-455c-9f89-14e2ce77040a admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:41:06.835024 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2182ea-6053-455c-9f89-14e2ce77040a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:41:06.835768 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2182ea-6053-455c-9f89-14e2ce77040a admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:41:06.846183 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1371/2747] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:41:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 11:41:06.850930 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7370df0a-ce8c-4914-a99d-5f7a69df6a45 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:41:06.860896 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7370df0a-ce8c-4914-a99d-5f7a69df6a45 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:41:06.865909 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cac37e-aedd-426b-8222-63eb0e05da1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:41:06.866786 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cac37e-aedd-426b-8222-63eb0e05da1e admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:41:06.867071 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cac37e-aedd-426b-8222-63eb0e05da1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:41:06.867350 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cac37e-aedd-426b-8222-63eb0e05da1e admin admin] PATH_INFO: `/v3/projects/40d5bbf32cb54de2a37895f53bcb9d4a/users/f9db0a0b5bcf4e6d8ee2593b21c415fb/roles/e1c88eebe911432cab6a07fbeb84653b` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:41:06.881756 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cac37e-aedd-426b-8222-63eb0e05da1e admin admin] RBAC: Authorizing `identity:create_grant(project_id=40d5bbf32cb54de2a37895f53bcb9d4a, user_id=f9db0a0b5bcf4e6d8ee2593b21c415fb, role_id=e1c88eebe911432cab6a07fbeb84653b)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:41:06.884424 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cac37e-aedd-426b-8222-63eb0e05da1e admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:41:06.984670 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1377/2748] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jun 29 11:41:06 2022] PUT /identity/v3/projects/40d5bbf32cb54de2a37895f53bcb9d4a/users/f9db0a0b5bcf4e6d8ee2593b21c415fb/roles/e1c88eebe911432cab6a07fbeb84653b => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:41:06.990042 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c510886-a743-47ee-b44e-67e55d211e02 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:41:06.990336 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c510886-a743-47ee-b44e-67e55d211e02 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:41:06.990623 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c510886-a743-47ee-b44e-67e55d211e02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:41:07.011701 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-3c510886-a743-47ee-b44e-67e55d211e02 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:41:07.013076 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-3c510886-a743-47ee-b44e-67e55d211e02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:41:07.074099 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1372/2749] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:41:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:41:07.084552 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cac37e-aedd-426b-8222-63eb0e05da1e admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:41:07.086548 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-91cac37e-aedd-426b-8222-63eb0e05da1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:41:07.101625 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:41:07.145010 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cac37e-aedd-426b-8222-63eb0e05da1e admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:41:07.148789 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63862e83-8777-4cff-a7e8-2ef012ce104c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f4bb9f989e4024a5da28dced646b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:41:07.152239 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63862e83-8777-4cff-a7e8-2ef012ce104c service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:41:07.152239 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63862e83-8777-4cff-a7e8-2ef012ce104c service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:41:07.152239 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63862e83-8777-4cff-a7e8-2ef012ce104c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:41:07.158894 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63862e83-8777-4cff-a7e8-2ef012ce104c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:41:07.160359 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63862e83-8777-4cff-a7e8-2ef012ce104c service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:41:07.169763 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1378/2750] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:41:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:41:08.153785 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c510886-a743-47ee-b44e-67e55d211e02 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:41:08.169592 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c510886-a743-47ee-b44e-67e55d211e02 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:41:08.174433 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd0c75b-fb17-49eb-83f5-1867122696b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f4bb9f989e4024a5da28dced646b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:41:08.175126 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd0c75b-fb17-49eb-83f5-1867122696b1 service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:41:08.176023 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd0c75b-fb17-49eb-83f5-1867122696b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:41:08.176350 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd0c75b-fb17-49eb-83f5-1867122696b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:41:08.178411 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-8dd0c75b-fb17-49eb-83f5-1867122696b1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:41:08.194273 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd0c75b-fb17-49eb-83f5-1867122696b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:41:08.195700 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd0c75b-fb17-49eb-83f5-1867122696b1 service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:41:08.206269 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1373/2751] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:41:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:41:12.711863 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63862e83-8777-4cff-a7e8-2ef012ce104c service neutron] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:41:12.714807 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-63862e83-8777-4cff-a7e8-2ef012ce104c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:41:12.732007 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63862e83-8777-4cff-a7e8-2ef012ce104c service neutron] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:41:12.740451 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354aa57f-db10-46ac-b621-4732e05eeca9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:41:12.742226 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354aa57f-db10-46ac-b621-4732e05eeca9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:41:12.744405 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354aa57f-db10-46ac-b621-4732e05eeca9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:41:12.744870 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354aa57f-db10-46ac-b621-4732e05eeca9 admin admin] PATH_INFO: `/v3/users/cf7bc4681c954c158a7051046bd9aa47` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:41:12.747868 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354aa57f-db10-46ac-b621-4732e05eeca9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf7bc4681c954c158a7051046bd9aa47)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:41:12.749785 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354aa57f-db10-46ac-b621-4732e05eeca9 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:41:13.042452 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-354aa57f-db10-46ac-b621-4732e05eeca9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf7bc4681c954c158a7051046bd9aa47'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:41:13.043192 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-354aa57f-db10-46ac-b621-4732e05eeca9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf7bc4681c954c158a7051046bd9aa47'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:41:13.115596 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-354aa57f-db10-46ac-b621-4732e05eeca9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf7bc4681c954c158a7051046bd9aa47'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:41:13.125104 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-354aa57f-db10-46ac-b621-4732e05eeca9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf7bc4681c954c158a7051046bd9aa47'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:41:13.125886 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-354aa57f-db10-46ac-b621-4732e05eeca9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf7bc4681c954c158a7051046bd9aa47'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:41:13.153697 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1379/2752] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jun 29 11:41:12 2022] DELETE /identity/v3/users/cf7bc4681c954c158a7051046bd9aa47 => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:41:13.437574 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd0c75b-fb17-49eb-83f5-1867122696b1 service neutron] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:41:13.451488 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd0c75b-fb17-49eb-83f5-1867122696b1 service neutron] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:41:13.528915 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47bdbbc-0eae-46f9-b5b5-cf2ab273c57b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:41:13.533155 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47bdbbc-0eae-46f9-b5b5-cf2ab273c57b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:41:13.533155 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47bdbbc-0eae-46f9-b5b5-cf2ab273c57b admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:41:13.533155 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47bdbbc-0eae-46f9-b5b5-cf2ab273c57b admin admin] PATH_INFO: `/v3/projects/7e39c9a1ff3c46dcbe4464f57534e5ea` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:41:13.533155 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47bdbbc-0eae-46f9-b5b5-cf2ab273c57b admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e39c9a1ff3c46dcbe4464f57534e5ea)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:41:13.533597 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47bdbbc-0eae-46f9-b5b5-cf2ab273c57b admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:41:13.627722 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-d47bdbbc-0eae-46f9-b5b5-cf2ab273c57b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e39c9a1ff3c46dcbe4464f57534e5ea', 'request_id': 'req-d47bdbbc-0eae-46f9-b5b5-cf2ab273c57b'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:41:13.633065 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=91,keystone:INSERT=4,keystone:DELETE=5 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:41:13.744282 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-d47bdbbc-0eae-46f9-b5b5-cf2ab273c57b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e39c9a1ff3c46dcbe4464f57534e5ea', 'request_id': 'req-d47bdbbc-0eae-46f9-b5b5-cf2ab273c57b'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:41:13.752954 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-d47bdbbc-0eae-46f9-b5b5-cf2ab273c57b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e39c9a1ff3c46dcbe4464f57534e5ea', 'request_id': 'req-d47bdbbc-0eae-46f9-b5b5-cf2ab273c57b'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:41:13.783145 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-d47bdbbc-0eae-46f9-b5b5-cf2ab273c57b admin admin] The token cache is being invalidate because project 7e39c9a1ff3c46dcbe4464f57534e5ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 11:41:13.786431 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-d47bdbbc-0eae-46f9-b5b5-cf2ab273c57b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:41:13.786431 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-d47bdbbc-0eae-46f9-b5b5-cf2ab273c57b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:41:13.786874 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1374/2753] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jun 29 11:41:13 2022] DELETE /identity/v3/projects/7e39c9a1ff3c46dcbe4464f57534e5ea => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:41:13.791189 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973f890c-d050-42e5-a02b-4739736c4cb7 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:41:13.792077 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973f890c-d050-42e5-a02b-4739736c4cb7 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:41:13.792521 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973f890c-d050-42e5-a02b-4739736c4cb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:41:13.819821 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-973f890c-d050-42e5-a02b-4739736c4cb7 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:41:13.821390 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-973f890c-d050-42e5-a02b-4739736c4cb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:41:13.908661 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1380/2754] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:41:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:41:13.921645 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e10e87-02c0-4794-ba3e-f8414589a32e None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:41:13.921932 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e10e87-02c0-4794-ba3e-f8414589a32e None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:41:13.922627 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e10e87-02c0-4794-ba3e-f8414589a32e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:41:13.949057 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-95e10e87-02c0-4794-ba3e-f8414589a32e None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:41:13.950587 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-95e10e87-02c0-4794-ba3e-f8414589a32e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:41:13.959845 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1375/2755] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:41:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:41:13.964256 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973f890c-d050-42e5-a02b-4739736c4cb7 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:41:13.976028 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973f890c-d050-42e5-a02b-4739736c4cb7 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:41:13.980887 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4172ea-35c4-4479-aa53-20db0b77a772 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:41:13.981564 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4172ea-35c4-4479-aa53-20db0b77a772 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:41:13.981849 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4172ea-35c4-4479-aa53-20db0b77a772 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:41:13.982124 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4172ea-35c4-4479-aa53-20db0b77a772 admin admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:41:13.982846 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4172ea-35c4-4479-aa53-20db0b77a772 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:41:13.983156 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4172ea-35c4-4479-aa53-20db0b77a772 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:41:13.984782 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4172ea-35c4-4479-aa53-20db0b77a772 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:41:14.001499 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1381/2756] 10.222.0.44 () {58 vars in 1152 bytes} [Wed Jun 29 11:41:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:41:14.003964 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e10e87-02c0-4794-ba3e-f8414589a32e None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:41:14.017463 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e10e87-02c0-4794-ba3e-f8414589a32e None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:41:14.028374 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c2c3a4-70af-4637-beb2-b11cab5852fa None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:41:14.028664 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c2c3a4-70af-4637-beb2-b11cab5852fa None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:41:14.028950 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c2c3a4-70af-4637-beb2-b11cab5852fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:41:14.030003 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0241cc5a-bdf4-422a-b990-962ab8e08c9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:41:14.030867 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0241cc5a-bdf4-422a-b990-962ab8e08c9c admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:41:14.031098 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0241cc5a-bdf4-422a-b990-962ab8e08c9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:41:14.031378 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0241cc5a-bdf4-422a-b990-962ab8e08c9c admin admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:41:14.032102 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0241cc5a-bdf4-422a-b990-962ab8e08c9c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:41:14.034031 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0241cc5a-bdf4-422a-b990-962ab8e08c9c admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:41:14.063910 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-99c2c3a4-70af-4637-beb2-b11cab5852fa None None] MFA Rules not processed for user `f9db0a0b5bcf4e6d8ee2593b21c415fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:41:14.063910 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-99c2c3a4-70af-4637-beb2-b11cab5852fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:41:14.067097 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:41:14.127943 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1382/2757] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:41:14 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:41:14.132441 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c2c3a4-70af-4637-beb2-b11cab5852fa None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:41:14.133749 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-99c2c3a4-70af-4637-beb2-b11cab5852fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:41:14.145519 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1376/2758] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jun 29 11:41:14 2022] POST /identity/v3/projects => generated 365 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:41:14.146762 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c2c3a4-70af-4637-beb2-b11cab5852fa None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:41:14.158360 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0241cc5a-bdf4-422a-b990-962ab8e08c9c admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:41:14.206865 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e065f26-a12d-45c2-9922-3830806e3dac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:41:14.207539 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e065f26-a12d-45c2-9922-3830806e3dac admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:41:14.207840 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e065f26-a12d-45c2-9922-3830806e3dac admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:41:14.208137 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e065f26-a12d-45c2-9922-3830806e3dac admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:41:14.208826 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e065f26-a12d-45c2-9922-3830806e3dac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:41:14.210758 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e065f26-a12d-45c2-9922-3830806e3dac admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:41:14.219744 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0241cc5a-bdf4-422a-b990-962ab8e08c9c admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:41:14.225388 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9817f028-da1c-46ea-ade2-3b97db40aeba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:41:14.226069 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9817f028-da1c-46ea-ade2-3b97db40aeba admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:41:14.226362 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9817f028-da1c-46ea-ade2-3b97db40aeba admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:41:14.227016 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9817f028-da1c-46ea-ade2-3b97db40aeba admin admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:41:14.227346 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9817f028-da1c-46ea-ade2-3b97db40aeba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:41:14.229903 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9817f028-da1c-46ea-ade2-3b97db40aeba admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:41:14.334442 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1383/2759] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jun 29 11:41:14 2022] POST /identity/v3/projects => generated 371 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 11:41:14.338323 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e065f26-a12d-45c2-9922-3830806e3dac admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:41:14.352935 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1377/2760] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jun 29 11:41:14 2022] POST /identity/v3/users => generated 283 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 11:41:14.354076 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9817f028-da1c-46ea-ade2-3b97db40aeba admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:41:14.364187 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9817f028-da1c-46ea-ade2-3b97db40aeba admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:41:14.403621 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e065f26-a12d-45c2-9922-3830806e3dac admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:41:14.408773 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19487fea-5332-46da-a15c-6c3db82658a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:41:14.410454 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19487fea-5332-46da-a15c-6c3db82658a2 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:41:14.410731 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19487fea-5332-46da-a15c-6c3db82658a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:41:14.411010 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19487fea-5332-46da-a15c-6c3db82658a2 admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:41:14.411720 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19487fea-5332-46da-a15c-6c3db82658a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:41:14.413682 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19487fea-5332-46da-a15c-6c3db82658a2 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:41:14.420954 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d34fbd4-5067-4fa0-aebc-7370cfe35495 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:41:14.421620 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d34fbd4-5067-4fa0-aebc-7370cfe35495 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:41:14.421907 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d34fbd4-5067-4fa0-aebc-7370cfe35495 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:41:14.422516 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d34fbd4-5067-4fa0-aebc-7370cfe35495 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:41:14.422905 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d34fbd4-5067-4fa0-aebc-7370cfe35495 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:41:14.424522 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d34fbd4-5067-4fa0-aebc-7370cfe35495 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:41:14.434352 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1378/2761] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:41:14 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 11:41:14.438436 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d34fbd4-5067-4fa0-aebc-7370cfe35495 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:41:14.450203 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d34fbd4-5067-4fa0-aebc-7370cfe35495 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:41:14.454956 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12738f36-c946-40ce-b51a-5c7144bf024f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:41:14.455785 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12738f36-c946-40ce-b51a-5c7144bf024f admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:41:14.456124 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12738f36-c946-40ce-b51a-5c7144bf024f admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:41:14.457001 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12738f36-c946-40ce-b51a-5c7144bf024f admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:41:14.457679 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12738f36-c946-40ce-b51a-5c7144bf024f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:41:14.459281 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12738f36-c946-40ce-b51a-5c7144bf024f admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:41:14.469971 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1379/2762] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:41:14 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 11:41:14.476739 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12738f36-c946-40ce-b51a-5c7144bf024f admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:41:14.486907 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12738f36-c946-40ce-b51a-5c7144bf024f admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:41:14.491856 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32e9ec3-fb72-44ae-9d0a-64d2b379d4f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:41:14.492760 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32e9ec3-fb72-44ae-9d0a-64d2b379d4f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:41:14.493049 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32e9ec3-fb72-44ae-9d0a-64d2b379d4f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:41:14.493319 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32e9ec3-fb72-44ae-9d0a-64d2b379d4f2 admin admin] PATH_INFO: `/v3/projects/156842ab4b8d4193adba44edf23da2a8/users/5dc6e28c7ed94c2f95a1b462cd9d4081/roles/e1c88eebe911432cab6a07fbeb84653b` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:41:14.509239 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32e9ec3-fb72-44ae-9d0a-64d2b379d4f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=156842ab4b8d4193adba44edf23da2a8, user_id=5dc6e28c7ed94c2f95a1b462cd9d4081, role_id=e1c88eebe911432cab6a07fbeb84653b)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:41:14.512126 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32e9ec3-fb72-44ae-9d0a-64d2b379d4f2 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:41:14.545110 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1384/2763] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jun 29 11:41:14 2022] POST /identity/v3/users => generated 292 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 11:41:14.549164 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19487fea-5332-46da-a15c-6c3db82658a2 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:41:14.559699 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19487fea-5332-46da-a15c-6c3db82658a2 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:41:14.564688 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39259a42-e111-4d05-b683-55367e2d08a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:41:14.565458 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39259a42-e111-4d05-b683-55367e2d08a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:41:14.565741 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39259a42-e111-4d05-b683-55367e2d08a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:41:14.566014 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39259a42-e111-4d05-b683-55367e2d08a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:41:14.566843 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39259a42-e111-4d05-b683-55367e2d08a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:41:14.568644 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39259a42-e111-4d05-b683-55367e2d08a4 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:41:14.579234 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1385/2764] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:41:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 11:41:14.584037 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39259a42-e111-4d05-b683-55367e2d08a4 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:41:14.594801 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39259a42-e111-4d05-b683-55367e2d08a4 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:41:14.600168 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a0218b-0cee-41c2-bd05-e94eb10632ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:41:14.601333 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a0218b-0cee-41c2-bd05-e94eb10632ec admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:41:14.601620 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a0218b-0cee-41c2-bd05-e94eb10632ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:41:14.602010 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a0218b-0cee-41c2-bd05-e94eb10632ec admin admin] PATH_INFO: `/v3/projects/7ec1045253fa42be8148bdeb29a8a1dc/users/d28eb9f00e1044888bbe8cc953247068/roles/e95c2ce63e114cc19a9dda6ad21e794f` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:41:14.605365 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:41:14.618662 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a0218b-0cee-41c2-bd05-e94eb10632ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ec1045253fa42be8148bdeb29a8a1dc, user_id=d28eb9f00e1044888bbe8cc953247068, role_id=e95c2ce63e114cc19a9dda6ad21e794f)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:41:14.618848 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1380/2765] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jun 29 11:41:14 2022] PUT /identity/v3/projects/156842ab4b8d4193adba44edf23da2a8/users/5dc6e28c7ed94c2f95a1b462cd9d4081/roles/e1c88eebe911432cab6a07fbeb84653b => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 11:41:14.624195 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a0218b-0cee-41c2-bd05-e94eb10632ec admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:41:14.629303 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef0f2a2-f19a-40c4-8c7f-173155818505 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:41:14.630289 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef0f2a2-f19a-40c4-8c7f-173155818505 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:41:14.630289 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef0f2a2-f19a-40c4-8c7f-173155818505 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:41:14.636741 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=93,keystone:DELETE=2,keystone:INSERT=5 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:41:14.657247 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-bef0f2a2-f19a-40c4-8c7f-173155818505 None None] MFA Rules not processed for user `5dc6e28c7ed94c2f95a1b462cd9d4081`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:41:14.658494 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-bef0f2a2-f19a-40c4-8c7f-173155818505 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:41:14.711462 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1386/2766] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jun 29 11:41:14 2022] PUT /identity/v3/projects/7ec1045253fa42be8148bdeb29a8a1dc/users/d28eb9f00e1044888bbe8cc953247068/roles/e95c2ce63e114cc19a9dda6ad21e794f => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:41:14.714982 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a0218b-0cee-41c2-bd05-e94eb10632ec admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:41:14.724066 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1381/2767] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:41:14 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 11:41:14.734835 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef0f2a2-f19a-40c4-8c7f-173155818505 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:41:14.736271 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-bef0f2a2-f19a-40c4-8c7f-173155818505 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:41:14.796278 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a0218b-0cee-41c2-bd05-e94eb10632ec admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:41:14.801913 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040b39eb-8045-47f6-9599-390ddcd5c2a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:41:14.802835 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040b39eb-8045-47f6-9599-390ddcd5c2a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:41:14.803304 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040b39eb-8045-47f6-9599-390ddcd5c2a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:41:14.803873 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040b39eb-8045-47f6-9599-390ddcd5c2a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:41:14.804814 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040b39eb-8045-47f6-9599-390ddcd5c2a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:41:14.806715 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040b39eb-8045-47f6-9599-390ddcd5c2a0 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:41:14.818185 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1387/2768] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:41:14 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 11:41:14.823483 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040b39eb-8045-47f6-9599-390ddcd5c2a0 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:41:14.823699 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef0f2a2-f19a-40c4-8c7f-173155818505 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:41:14.836049 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040b39eb-8045-47f6-9599-390ddcd5c2a0 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:41:14.841192 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbce2e11-8cdd-444c-afc3-fef0ddcdf745 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:41:14.842255 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbce2e11-8cdd-444c-afc3-fef0ddcdf745 admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:41:14.842732 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbce2e11-8cdd-444c-afc3-fef0ddcdf745 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:41:14.843010 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbce2e11-8cdd-444c-afc3-fef0ddcdf745 admin admin] PATH_INFO: `/v3/projects/7ec1045253fa42be8148bdeb29a8a1dc/users/d28eb9f00e1044888bbe8cc953247068/roles/e95c2ce63e114cc19a9dda6ad21e794f` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:41:14.845019 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbce2e11-8cdd-444c-afc3-fef0ddcdf745 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ec1045253fa42be8148bdeb29a8a1dc, user_id=d28eb9f00e1044888bbe8cc953247068, role_id=e95c2ce63e114cc19a9dda6ad21e794f)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:41:14.847878 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbce2e11-8cdd-444c-afc3-fef0ddcdf745 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:41:14.861205 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1388/2769] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jun 29 11:41:14 2022] PUT /identity/v3/projects/7ec1045253fa42be8148bdeb29a8a1dc/users/d28eb9f00e1044888bbe8cc953247068/roles/e95c2ce63e114cc19a9dda6ad21e794f => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:41:14.881093 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e243f47-3c1b-40f1-88ee-0d61e91b4a24 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff7931b75a924394b1af733febce0834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:41:14.885914 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e243f47-3c1b-40f1-88ee-0d61e91b4a24 service cinder] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:41:14.885914 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e243f47-3c1b-40f1-88ee-0d61e91b4a24 service cinder] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:41:14.885914 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e243f47-3c1b-40f1-88ee-0d61e91b4a24 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:41:14.931737 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:41:14.945099 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e243f47-3c1b-40f1-88ee-0d61e91b4a24 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:41:14.946563 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e243f47-3c1b-40f1-88ee-0d61e91b4a24 service cinder] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:41:14.956373 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1382/2770] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:41:14 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 11:41:20.488388 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88220a08-c2eb-4c51-bf6c-c1b94bb777c6 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:41:20.488388 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88220a08-c2eb-4c51-bf6c-c1b94bb777c6 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:41:20.488388 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88220a08-c2eb-4c51-bf6c-c1b94bb777c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:41:20.516246 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-88220a08-c2eb-4c51-bf6c-c1b94bb777c6 None None] MFA Rules not processed for user `d28eb9f00e1044888bbe8cc953247068`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:41:20.516246 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-88220a08-c2eb-4c51-bf6c-c1b94bb777c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:41:20.591528 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1389/2771] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:41:20 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:41:20.607291 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e243f47-3c1b-40f1-88ee-0d61e91b4a24 service cinder] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:41:20.608735 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-1e243f47-3c1b-40f1-88ee-0d61e91b4a24 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:41:20.683286 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e243f47-3c1b-40f1-88ee-0d61e91b4a24 service cinder] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:41:20.746846 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65e579c-9fac-4c4b-bc5c-1ab2339aaddd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '796aeeff16df4739bddec1f6dee61bfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:41:20.747503 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65e579c-9fac-4c4b-bc5c-1ab2339aaddd service nova] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:41:20.747916 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65e579c-9fac-4c4b-bc5c-1ab2339aaddd service nova] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:41:20.748385 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65e579c-9fac-4c4b-bc5c-1ab2339aaddd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:41:20.806622 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65e579c-9fac-4c4b-bc5c-1ab2339aaddd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:41:20.808256 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65e579c-9fac-4c4b-bc5c-1ab2339aaddd service nova] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:41:20.817700 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1383/2772] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:41:20 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:41:23.678837 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034065f4-0bae-4dda-8301-93c8fcd3e9d2 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:41:23.679384 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034065f4-0bae-4dda-8301-93c8fcd3e9d2 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:41:23.679488 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034065f4-0bae-4dda-8301-93c8fcd3e9d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:41:23.697606 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-034065f4-0bae-4dda-8301-93c8fcd3e9d2 None None] MFA Rules not processed for user `f9db0a0b5bcf4e6d8ee2593b21c415fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:41:23.698805 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-034065f4-0bae-4dda-8301-93c8fcd3e9d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:41:23.705674 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1390/2773] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:41:23 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:41:26.220552 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65e579c-9fac-4c4b-bc5c-1ab2339aaddd service nova] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:41:26.243283 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=100 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:41:26.288461 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65e579c-9fac-4c4b-bc5c-1ab2339aaddd service nova] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:41:26.293212 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b765dff6-d78d-46a9-9733-204a2f4d580f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:41:26.294092 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b765dff6-d78d-46a9-9733-204a2f4d580f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:41:26.294370 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b765dff6-d78d-46a9-9733-204a2f4d580f admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:41:26.294645 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b765dff6-d78d-46a9-9733-204a2f4d580f admin admin] PATH_INFO: `/v3/users/5dc6e28c7ed94c2f95a1b462cd9d4081` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:41:26.295858 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b765dff6-d78d-46a9-9733-204a2f4d580f admin admin] RBAC: Authorizing `identity:delete_user(user_id=5dc6e28c7ed94c2f95a1b462cd9d4081)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:41:26.297784 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b765dff6-d78d-46a9-9733-204a2f4d580f admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:41:26.421591 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-b765dff6-d78d-46a9-9733-204a2f4d580f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5dc6e28c7ed94c2f95a1b462cd9d4081'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:41:26.469760 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-b765dff6-d78d-46a9-9733-204a2f4d580f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5dc6e28c7ed94c2f95a1b462cd9d4081'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:41:26.470374 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-b765dff6-d78d-46a9-9733-204a2f4d580f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5dc6e28c7ed94c2f95a1b462cd9d4081'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:41:26.487703 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-b765dff6-d78d-46a9-9733-204a2f4d580f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5dc6e28c7ed94c2f95a1b462cd9d4081'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:41:26.488476 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-b765dff6-d78d-46a9-9733-204a2f4d580f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5dc6e28c7ed94c2f95a1b462cd9d4081'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:41:26.501367 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1384/2774] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jun 29 11:41:26 2022] DELETE /identity/v3/users/5dc6e28c7ed94c2f95a1b462cd9d4081 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:41:26.505787 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bb0fed-876b-4ef6-a202-e64d49f83451 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:41:26.506284 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bb0fed-876b-4ef6-a202-e64d49f83451 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:41:26.506410 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bb0fed-876b-4ef6-a202-e64d49f83451 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:41:26.525585 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-54bb0fed-876b-4ef6-a202-e64d49f83451 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:41:26.526809 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-54bb0fed-876b-4ef6-a202-e64d49f83451 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:41:26.570829 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:41:26.588518 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1391/2775] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:41:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:41:26.601922 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b765dff6-d78d-46a9-9733-204a2f4d580f admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:41:26.604866 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-b765dff6-d78d-46a9-9733-204a2f4d580f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:41:26.671294 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b765dff6-d78d-46a9-9733-204a2f4d580f admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:41:26.675397 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775a08e2-b4ed-437a-b293-67f43f20ebc0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f4bb9f989e4024a5da28dced646b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:41:26.676309 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775a08e2-b4ed-437a-b293-67f43f20ebc0 service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:41:26.676664 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775a08e2-b4ed-437a-b293-67f43f20ebc0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:41:26.676951 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775a08e2-b4ed-437a-b293-67f43f20ebc0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:41:26.687183 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775a08e2-b4ed-437a-b293-67f43f20ebc0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:41:26.688661 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775a08e2-b4ed-437a-b293-67f43f20ebc0 service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:41:26.698840 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1385/2776] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:41:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:41:27.102488 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bb0fed-876b-4ef6-a202-e64d49f83451 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:41:27.104127 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-54bb0fed-876b-4ef6-a202-e64d49f83451 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:41:27.117219 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bb0fed-876b-4ef6-a202-e64d49f83451 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:41:27.123297 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49fd3e4-23f7-4d55-ace2-9f09cc4a9b45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:41:27.124492 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49fd3e4-23f7-4d55-ace2-9f09cc4a9b45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:41:27.124967 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49fd3e4-23f7-4d55-ace2-9f09cc4a9b45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:41:27.125420 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49fd3e4-23f7-4d55-ace2-9f09cc4a9b45 admin admin] PATH_INFO: `/v3/projects/156842ab4b8d4193adba44edf23da2a8` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:41:27.126954 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49fd3e4-23f7-4d55-ace2-9f09cc4a9b45 admin admin] RBAC: Authorizing `identity:delete_project(project_id=156842ab4b8d4193adba44edf23da2a8)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:41:27.129575 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49fd3e4-23f7-4d55-ace2-9f09cc4a9b45 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:41:27.239853 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-a49fd3e4-23f7-4d55-ace2-9f09cc4a9b45 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '156842ab4b8d4193adba44edf23da2a8', 'request_id': 'req-a49fd3e4-23f7-4d55-ace2-9f09cc4a9b45'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:41:27.246370 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-a49fd3e4-23f7-4d55-ace2-9f09cc4a9b45 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '156842ab4b8d4193adba44edf23da2a8', 'request_id': 'req-a49fd3e4-23f7-4d55-ace2-9f09cc4a9b45'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:41:27.251955 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-a49fd3e4-23f7-4d55-ace2-9f09cc4a9b45 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '156842ab4b8d4193adba44edf23da2a8', 'request_id': 'req-a49fd3e4-23f7-4d55-ace2-9f09cc4a9b45'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:41:27.349288 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-a49fd3e4-23f7-4d55-ace2-9f09cc4a9b45 admin admin] The token cache is being invalidate because project 156842ab4b8d4193adba44edf23da2a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 11:41:27.349560 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-a49fd3e4-23f7-4d55-ace2-9f09cc4a9b45 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:41:27.350308 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-a49fd3e4-23f7-4d55-ace2-9f09cc4a9b45 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:41:27.352331 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1392/2777] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jun 29 11:41:27 2022] DELETE /identity/v3/projects/156842ab4b8d4193adba44edf23da2a8 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:41:27.357348 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4593863-2466-4b8f-a83e-f0d51feed7bd None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:41:27.357915 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4593863-2466-4b8f-a83e-f0d51feed7bd None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:41:27.357915 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4593863-2466-4b8f-a83e-f0d51feed7bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:41:27.376012 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-d4593863-2466-4b8f-a83e-f0d51feed7bd None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:41:27.377209 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-d4593863-2466-4b8f-a83e-f0d51feed7bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:41:27.418692 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:41:27.442874 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1386/2778] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:41:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:41:27.450574 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6915d525-83bd-4b37-997c-b3b6e567d211 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:41:27.450864 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6915d525-83bd-4b37-997c-b3b6e567d211 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:41:27.451163 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6915d525-83bd-4b37-997c-b3b6e567d211 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:41:27.469354 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-6915d525-83bd-4b37-997c-b3b6e567d211 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:41:27.470547 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-6915d525-83bd-4b37-997c-b3b6e567d211 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:41:27.480288 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1393/2779] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:41:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:41:27.484605 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4593863-2466-4b8f-a83e-f0d51feed7bd None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:41:27.495109 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4593863-2466-4b8f-a83e-f0d51feed7bd None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:41:27.500098 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f381ba-942e-4bad-8661-14061be6f387 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:41:27.500747 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f381ba-942e-4bad-8661-14061be6f387 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:41:27.501031 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f381ba-942e-4bad-8661-14061be6f387 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:41:27.501305 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f381ba-942e-4bad-8661-14061be6f387 admin admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:41:27.502059 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f381ba-942e-4bad-8661-14061be6f387 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:41:27.502367 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f381ba-942e-4bad-8661-14061be6f387 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:41:27.504022 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f381ba-942e-4bad-8661-14061be6f387 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:41:27.516581 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1387/2780] 10.222.0.44 () {58 vars in 1152 bytes} [Wed Jun 29 11:41:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 11:41:27.521577 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6915d525-83bd-4b37-997c-b3b6e567d211 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:41:27.530902 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6915d525-83bd-4b37-997c-b3b6e567d211 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:41:27.536075 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3371e9c2-d6f3-4240-992d-af54609569e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:41:27.536726 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3371e9c2-d6f3-4240-992d-af54609569e9 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:41:27.537008 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3371e9c2-d6f3-4240-992d-af54609569e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:41:27.537280 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3371e9c2-d6f3-4240-992d-af54609569e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:41:27.537962 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3371e9c2-d6f3-4240-992d-af54609569e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:41:27.539849 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3371e9c2-d6f3-4240-992d-af54609569e9 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:41:27.625598 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1394/2781] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jun 29 11:41:27 2022] POST /identity/v3/projects => generated 403 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 11:41:27.629686 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f381ba-942e-4bad-8661-14061be6f387 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:41:27.640167 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f381ba-942e-4bad-8661-14061be6f387 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:41:27.700887 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9187b62-a1ee-4d65-8c0e-3a78df4bf0f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:41:27.701562 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9187b62-a1ee-4d65-8c0e-3a78df4bf0f9 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:41:27.701849 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9187b62-a1ee-4d65-8c0e-3a78df4bf0f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:41:27.702127 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9187b62-a1ee-4d65-8c0e-3a78df4bf0f9 admin admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:41:27.702867 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9187b62-a1ee-4d65-8c0e-3a78df4bf0f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:41:27.704836 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9187b62-a1ee-4d65-8c0e-3a78df4bf0f9 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:41:27.785097 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1388/2782] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jun 29 11:41:27 2022] POST /identity/v3/users => generated 302 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:41:27.789401 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3371e9c2-d6f3-4240-992d-af54609569e9 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:41:27.800875 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3371e9c2-d6f3-4240-992d-af54609569e9 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:41:27.805761 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cf7780-59ce-4d9a-a619-44df0e7949bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:41:27.806428 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cf7780-59ce-4d9a-a619-44df0e7949bc admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:41:27.806731 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cf7780-59ce-4d9a-a619-44df0e7949bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:41:27.807004 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cf7780-59ce-4d9a-a619-44df0e7949bc admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:41:27.807727 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cf7780-59ce-4d9a-a619-44df0e7949bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:41:27.809363 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cf7780-59ce-4d9a-a619-44df0e7949bc admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:41:27.819752 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1395/2783] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:41:27 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 11:41:27.833872 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9187b62-a1ee-4d65-8c0e-3a78df4bf0f9 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:41:27.844431 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9187b62-a1ee-4d65-8c0e-3a78df4bf0f9 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:41:27.849655 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fe479b-6b91-4356-99bc-5a288ba2877d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:41:27.850310 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fe479b-6b91-4356-99bc-5a288ba2877d admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:41:27.850623 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fe479b-6b91-4356-99bc-5a288ba2877d admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:41:27.850900 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fe479b-6b91-4356-99bc-5a288ba2877d admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:41:27.851625 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fe479b-6b91-4356-99bc-5a288ba2877d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:41:27.853329 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fe479b-6b91-4356-99bc-5a288ba2877d admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:41:27.863314 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1389/2784] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:41:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 11:41:27.867824 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cf7780-59ce-4d9a-a619-44df0e7949bc admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:41:27.878732 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cf7780-59ce-4d9a-a619-44df0e7949bc admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:41:27.883421 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b55bfed-71bd-4e45-8d93-c302aff96ba1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:41:27.884341 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b55bfed-71bd-4e45-8d93-c302aff96ba1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:41:27.884649 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b55bfed-71bd-4e45-8d93-c302aff96ba1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:41:27.884908 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b55bfed-71bd-4e45-8d93-c302aff96ba1 admin admin] PATH_INFO: `/v3/projects/57870a12b34a41f095a3fe2ece303e14/users/d5255d0972424593b074a4e494b1f604/roles/e1c88eebe911432cab6a07fbeb84653b` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:41:27.900177 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b55bfed-71bd-4e45-8d93-c302aff96ba1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=57870a12b34a41f095a3fe2ece303e14, user_id=d5255d0972424593b074a4e494b1f604, role_id=e1c88eebe911432cab6a07fbeb84653b)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:41:27.902971 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b55bfed-71bd-4e45-8d93-c302aff96ba1 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:41:28.001891 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1396/2785] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jun 29 11:41:27 2022] PUT /identity/v3/projects/57870a12b34a41f095a3fe2ece303e14/users/d5255d0972424593b074a4e494b1f604/roles/e1c88eebe911432cab6a07fbeb84653b => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:41:28.014116 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c315d50-6a61-4b0d-b9c2-aa02ec64fe64 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:41:28.014408 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c315d50-6a61-4b0d-b9c2-aa02ec64fe64 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:41:28.014687 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c315d50-6a61-4b0d-b9c2-aa02ec64fe64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:41:28.041140 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-9c315d50-6a61-4b0d-b9c2-aa02ec64fe64 None None] MFA Rules not processed for user `d5255d0972424593b074a4e494b1f604`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:41:28.042390 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-9c315d50-6a61-4b0d-b9c2-aa02ec64fe64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:41:28.107130 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1390/2786] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:41:28 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:41:28.111637 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b55bfed-71bd-4e45-8d93-c302aff96ba1 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:41:28.177812 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b55bfed-71bd-4e45-8d93-c302aff96ba1 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:41:28.182439 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d696ec9-8d62-4626-b3f3-eb3088bdd7ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:41:28.183110 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d696ec9-8d62-4626-b3f3-eb3088bdd7ed admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:41:28.183398 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d696ec9-8d62-4626-b3f3-eb3088bdd7ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:41:28.183680 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d696ec9-8d62-4626-b3f3-eb3088bdd7ed admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:41:28.184402 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d696ec9-8d62-4626-b3f3-eb3088bdd7ed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:41:28.186289 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d696ec9-8d62-4626-b3f3-eb3088bdd7ed admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:41:28.196102 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:41:28.265214 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1397/2787] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jun 29 11:41:28 2022] POST /identity/v3/projects => generated 405 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:41:28.269614 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c315d50-6a61-4b0d-b9c2-aa02ec64fe64 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:41:28.280987 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c315d50-6a61-4b0d-b9c2-aa02ec64fe64 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:41:28.316715 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:41:28.339405 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f72eb2d-bda8-4ff8-bfa8-082ce3b6869c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:41:28.340121 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f72eb2d-bda8-4ff8-bfa8-082ce3b6869c admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:41:28.340413 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f72eb2d-bda8-4ff8-bfa8-082ce3b6869c admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:41:28.340688 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f72eb2d-bda8-4ff8-bfa8-082ce3b6869c admin admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:41:28.341400 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f72eb2d-bda8-4ff8-bfa8-082ce3b6869c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:41:28.343331 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f72eb2d-bda8-4ff8-bfa8-082ce3b6869c admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:41:28.424200 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1391/2788] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jun 29 11:41:28 2022] POST /identity/v3/users => generated 309 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 11:41:28.428872 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d696ec9-8d62-4626-b3f3-eb3088bdd7ed admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:41:28.439440 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d696ec9-8d62-4626-b3f3-eb3088bdd7ed admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:41:28.444830 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866ddd78-cfcb-4e19-811e-893a5745d9fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:41:28.445816 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866ddd78-cfcb-4e19-811e-893a5745d9fc admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:41:28.446211 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866ddd78-cfcb-4e19-811e-893a5745d9fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:41:28.446496 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866ddd78-cfcb-4e19-811e-893a5745d9fc admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:41:28.447349 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866ddd78-cfcb-4e19-811e-893a5745d9fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:41:28.449057 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866ddd78-cfcb-4e19-811e-893a5745d9fc admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:41:28.460370 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1398/2789] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:41:28 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 11:41:28.464679 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f72eb2d-bda8-4ff8-bfa8-082ce3b6869c admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:41:28.475538 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f72eb2d-bda8-4ff8-bfa8-082ce3b6869c admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:41:28.480250 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f12e7e-7b03-40b7-b7db-8a02736918f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:41:28.481118 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f12e7e-7b03-40b7-b7db-8a02736918f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:41:28.481403 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f12e7e-7b03-40b7-b7db-8a02736918f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:41:28.481680 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f12e7e-7b03-40b7-b7db-8a02736918f6 admin admin] PATH_INFO: `/v3/projects/ceb5aa373b944047a118cd78dcb50d90/users/d2b46fcbb2ca47a2a92d379b5a2facc1/roles/e95c2ce63e114cc19a9dda6ad21e794f` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:41:28.498512 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f12e7e-7b03-40b7-b7db-8a02736918f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ceb5aa373b944047a118cd78dcb50d90, user_id=d2b46fcbb2ca47a2a92d379b5a2facc1, role_id=e95c2ce63e114cc19a9dda6ad21e794f)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:41:28.501207 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f12e7e-7b03-40b7-b7db-8a02736918f6 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:41:28.572941 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1392/2790] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jun 29 11:41:28 2022] PUT /identity/v3/projects/ceb5aa373b944047a118cd78dcb50d90/users/d2b46fcbb2ca47a2a92d379b5a2facc1/roles/e95c2ce63e114cc19a9dda6ad21e794f => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 11:41:28.577183 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866ddd78-cfcb-4e19-811e-893a5745d9fc admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:41:28.591052 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866ddd78-cfcb-4e19-811e-893a5745d9fc admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:41:28.650965 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc66710-86e4-4d6c-b9cb-ee6e705d32ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:41:28.651632 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc66710-86e4-4d6c-b9cb-ee6e705d32ac admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:41:28.651943 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc66710-86e4-4d6c-b9cb-ee6e705d32ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:41:28.652218 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc66710-86e4-4d6c-b9cb-ee6e705d32ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:41:28.652927 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc66710-86e4-4d6c-b9cb-ee6e705d32ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:41:28.654491 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc66710-86e4-4d6c-b9cb-ee6e705d32ac admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:41:28.667688 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1399/2791] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:41:28 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 11:41:28.670773 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f12e7e-7b03-40b7-b7db-8a02736918f6 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:41:28.681382 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f12e7e-7b03-40b7-b7db-8a02736918f6 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:41:28.686422 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ac6976-fd0d-4362-a6f2-e8883a3740e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:41:28.687297 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ac6976-fd0d-4362-a6f2-e8883a3740e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:41:28.687580 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ac6976-fd0d-4362-a6f2-e8883a3740e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:41:28.687882 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ac6976-fd0d-4362-a6f2-e8883a3740e4 admin admin] PATH_INFO: `/v3/projects/ceb5aa373b944047a118cd78dcb50d90/users/d2b46fcbb2ca47a2a92d379b5a2facc1/roles/e95c2ce63e114cc19a9dda6ad21e794f` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:41:28.689846 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ac6976-fd0d-4362-a6f2-e8883a3740e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ceb5aa373b944047a118cd78dcb50d90, user_id=d2b46fcbb2ca47a2a92d379b5a2facc1, role_id=e95c2ce63e114cc19a9dda6ad21e794f)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:41:28.692936 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ac6976-fd0d-4362-a6f2-e8883a3740e4 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:41:28.703163 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1393/2792] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jun 29 11:41:28 2022] PUT /identity/v3/projects/ceb5aa373b944047a118cd78dcb50d90/users/d2b46fcbb2ca47a2a92d379b5a2facc1/roles/e95c2ce63e114cc19a9dda6ad21e794f => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:41:28.713924 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c91846-2169-41aa-a04e-97abcdf2be9c None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:41:28.714247 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c91846-2169-41aa-a04e-97abcdf2be9c None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:41:28.714536 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c91846-2169-41aa-a04e-97abcdf2be9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:41:28.742346 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-a7c91846-2169-41aa-a04e-97abcdf2be9c None None] MFA Rules not processed for user `d2b46fcbb2ca47a2a92d379b5a2facc1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:41:28.743551 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-a7c91846-2169-41aa-a04e-97abcdf2be9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:41:28.814011 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1400/2793] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:41:28 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:41:28.825544 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ac6976-fd0d-4362-a6f2-e8883a3740e4 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:41:28.827311 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-28ac6976-fd0d-4362-a6f2-e8883a3740e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:41:28.884733 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ac6976-fd0d-4362-a6f2-e8883a3740e4 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:41:28.888388 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7529bd-843f-4934-9b2b-c737a4d59de4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff7931b75a924394b1af733febce0834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:41:28.889030 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7529bd-843f-4934-9b2b-c737a4d59de4 service cinder] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:41:28.889309 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7529bd-843f-4934-9b2b-c737a4d59de4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:41:28.889584 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7529bd-843f-4934-9b2b-c737a4d59de4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:41:28.898420 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7529bd-843f-4934-9b2b-c737a4d59de4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:41:28.899972 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7529bd-843f-4934-9b2b-c737a4d59de4 service cinder] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:41:28.910755 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1394/2794] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:41:28 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:41:29.057630 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c91846-2169-41aa-a04e-97abcdf2be9c None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:41:29.120973 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c91846-2169-41aa-a04e-97abcdf2be9c None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:41:29.125596 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f5c1d1-6f47-484b-875a-c1ba27e98ede admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:41:29.126492 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f5c1d1-6f47-484b-875a-c1ba27e98ede admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:41:29.126794 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f5c1d1-6f47-484b-875a-c1ba27e98ede admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:41:29.127078 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f5c1d1-6f47-484b-875a-c1ba27e98ede admin admin] PATH_INFO: `/v3/users/d5255d0972424593b074a4e494b1f604` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:41:29.128435 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f5c1d1-6f47-484b-875a-c1ba27e98ede admin admin] RBAC: Authorizing `identity:delete_user(user_id=d5255d0972424593b074a4e494b1f604)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:41:29.130842 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f5c1d1-6f47-484b-875a-c1ba27e98ede admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:41:29.133135 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:41:29.329811 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-52f5c1d1-6f47-484b-875a-c1ba27e98ede admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd5255d0972424593b074a4e494b1f604'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:41:29.330498 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-52f5c1d1-6f47-484b-875a-c1ba27e98ede admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd5255d0972424593b074a4e494b1f604'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:41:29.394895 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-52f5c1d1-6f47-484b-875a-c1ba27e98ede admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd5255d0972424593b074a4e494b1f604'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:41:29.403340 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-52f5c1d1-6f47-484b-875a-c1ba27e98ede admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd5255d0972424593b074a4e494b1f604'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:41:29.404091 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-52f5c1d1-6f47-484b-875a-c1ba27e98ede admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd5255d0972424593b074a4e494b1f604'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:41:29.422305 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1401/2795] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jun 29 11:41:29 2022] DELETE /identity/v3/users/d5255d0972424593b074a4e494b1f604 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:41:29.427344 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97570e2a-c1b9-493d-9aa1-40b10add65f5 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:41:29.427881 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97570e2a-c1b9-493d-9aa1-40b10add65f5 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:41:29.428008 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97570e2a-c1b9-493d-9aa1-40b10add65f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:41:29.446270 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-97570e2a-c1b9-493d-9aa1-40b10add65f5 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:41:29.447492 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-97570e2a-c1b9-493d-9aa1-40b10add65f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:41:29.508281 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1395/2796] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:41:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:41:29.518317 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f5c1d1-6f47-484b-875a-c1ba27e98ede admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:41:29.519804 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-52f5c1d1-6f47-484b-875a-c1ba27e98ede admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:41:29.578403 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f5c1d1-6f47-484b-875a-c1ba27e98ede admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:41:29.582293 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de1d542-d177-4d13-8293-9e5fcc3c26b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f4bb9f989e4024a5da28dced646b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:41:29.582884 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de1d542-d177-4d13-8293-9e5fcc3c26b6 service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:41:29.583169 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de1d542-d177-4d13-8293-9e5fcc3c26b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:41:29.583450 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de1d542-d177-4d13-8293-9e5fcc3c26b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:41:29.592472 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de1d542-d177-4d13-8293-9e5fcc3c26b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:41:29.593961 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de1d542-d177-4d13-8293-9e5fcc3c26b6 service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:41:29.604965 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1402/2797] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:41:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:41:30.088289 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97570e2a-c1b9-493d-9aa1-40b10add65f5 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:41:30.089722 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-97570e2a-c1b9-493d-9aa1-40b10add65f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:41:30.108353 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97570e2a-c1b9-493d-9aa1-40b10add65f5 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:41:30.113272 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c8a362-b5c8-4dd7-b13a-1e5c4159dca8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:41:30.114150 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c8a362-b5c8-4dd7-b13a-1e5c4159dca8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:41:30.114461 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c8a362-b5c8-4dd7-b13a-1e5c4159dca8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:41:30.114788 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c8a362-b5c8-4dd7-b13a-1e5c4159dca8 admin admin] PATH_INFO: `/v3/projects/57870a12b34a41f095a3fe2ece303e14` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:41:30.116131 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c8a362-b5c8-4dd7-b13a-1e5c4159dca8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=57870a12b34a41f095a3fe2ece303e14)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:41:30.117850 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c8a362-b5c8-4dd7-b13a-1e5c4159dca8 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:41:30.202372 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=93,keystone:INSERT=5,keystone:DELETE=2 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:41:30.217623 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-11c8a362-b5c8-4dd7-b13a-1e5c4159dca8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '57870a12b34a41f095a3fe2ece303e14', 'request_id': 'req-11c8a362-b5c8-4dd7-b13a-1e5c4159dca8'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:41:30.297194 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-11c8a362-b5c8-4dd7-b13a-1e5c4159dca8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '57870a12b34a41f095a3fe2ece303e14', 'request_id': 'req-11c8a362-b5c8-4dd7-b13a-1e5c4159dca8'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:41:30.304255 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-11c8a362-b5c8-4dd7-b13a-1e5c4159dca8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '57870a12b34a41f095a3fe2ece303e14', 'request_id': 'req-11c8a362-b5c8-4dd7-b13a-1e5c4159dca8'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:41:30.314835 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-11c8a362-b5c8-4dd7-b13a-1e5c4159dca8 admin admin] The token cache is being invalidate because project 57870a12b34a41f095a3fe2ece303e14 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 11:41:30.315132 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-11c8a362-b5c8-4dd7-b13a-1e5c4159dca8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:41:30.315796 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-11c8a362-b5c8-4dd7-b13a-1e5c4159dca8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:41:30.317361 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1396/2798] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jun 29 11:41:30 2022] DELETE /identity/v3/projects/57870a12b34a41f095a3fe2ece303e14 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:41:30.321330 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de1d542-d177-4d13-8293-9e5fcc3c26b6 service neutron] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:41:30.331792 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de1d542-d177-4d13-8293-9e5fcc3c26b6 service neutron] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:41:30.390661 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a9d37f-25d7-43fc-8854-ac391895b5f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:41:30.391443 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a9d37f-25d7-43fc-8854-ac391895b5f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:41:30.391863 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a9d37f-25d7-43fc-8854-ac391895b5f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:41:30.392148 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a9d37f-25d7-43fc-8854-ac391895b5f1 admin admin] PATH_INFO: `/v3/users/d2b46fcbb2ca47a2a92d379b5a2facc1` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:41:30.393349 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a9d37f-25d7-43fc-8854-ac391895b5f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d2b46fcbb2ca47a2a92d379b5a2facc1)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:41:30.395895 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a9d37f-25d7-43fc-8854-ac391895b5f1 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:41:30.510725 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:41:30.638138 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-a2a9d37f-25d7-43fc-8854-ac391895b5f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd2b46fcbb2ca47a2a92d379b5a2facc1'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:41:30.638883 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-a2a9d37f-25d7-43fc-8854-ac391895b5f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd2b46fcbb2ca47a2a92d379b5a2facc1'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:41:30.720977 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-a2a9d37f-25d7-43fc-8854-ac391895b5f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd2b46fcbb2ca47a2a92d379b5a2facc1'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:41:30.729059 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-a2a9d37f-25d7-43fc-8854-ac391895b5f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd2b46fcbb2ca47a2a92d379b5a2facc1'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:41:30.730665 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-a2a9d37f-25d7-43fc-8854-ac391895b5f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd2b46fcbb2ca47a2a92d379b5a2facc1'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:41:30.747288 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1403/2799] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jun 29 11:41:30 2022] DELETE /identity/v3/users/d2b46fcbb2ca47a2a92d379b5a2facc1 => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:41:31.151630 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c8a362-b5c8-4dd7-b13a-1e5c4159dca8 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:41:31.153391 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-11c8a362-b5c8-4dd7-b13a-1e5c4159dca8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:41:31.167714 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c8a362-b5c8-4dd7-b13a-1e5c4159dca8 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:41:31.227074 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab8e36f-4ce3-40e9-8708-5a4fef3a9fe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:41:31.228016 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab8e36f-4ce3-40e9-8708-5a4fef3a9fe6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:41:31.228303 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab8e36f-4ce3-40e9-8708-5a4fef3a9fe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:41:31.228579 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab8e36f-4ce3-40e9-8708-5a4fef3a9fe6 admin admin] PATH_INFO: `/v3/projects/ceb5aa373b944047a118cd78dcb50d90` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:41:31.229767 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab8e36f-4ce3-40e9-8708-5a4fef3a9fe6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ceb5aa373b944047a118cd78dcb50d90)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:41:31.231735 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab8e36f-4ce3-40e9-8708-5a4fef3a9fe6 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:41:31.333565 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-2ab8e36f-4ce3-40e9-8708-5a4fef3a9fe6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ceb5aa373b944047a118cd78dcb50d90', 'request_id': 'req-2ab8e36f-4ce3-40e9-8708-5a4fef3a9fe6'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:41:31.400698 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-2ab8e36f-4ce3-40e9-8708-5a4fef3a9fe6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ceb5aa373b944047a118cd78dcb50d90', 'request_id': 'req-2ab8e36f-4ce3-40e9-8708-5a4fef3a9fe6'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:41:31.407544 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-2ab8e36f-4ce3-40e9-8708-5a4fef3a9fe6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ceb5aa373b944047a118cd78dcb50d90', 'request_id': 'req-2ab8e36f-4ce3-40e9-8708-5a4fef3a9fe6'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:41:31.418789 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-2ab8e36f-4ce3-40e9-8708-5a4fef3a9fe6 admin admin] The token cache is being invalidate because project ceb5aa373b944047a118cd78dcb50d90 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 11:41:31.419080 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-2ab8e36f-4ce3-40e9-8708-5a4fef3a9fe6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:41:31.419715 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-2ab8e36f-4ce3-40e9-8708-5a4fef3a9fe6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:41:31.421289 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1397/2800] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jun 29 11:41:31 2022] DELETE /identity/v3/projects/ceb5aa373b944047a118cd78dcb50d90 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:41:31.427329 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf13a20-41e3-4025-b64f-fe75c4798a9b None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:41:31.427645 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf13a20-41e3-4025-b64f-fe75c4798a9b None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:41:31.427896 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf13a20-41e3-4025-b64f-fe75c4798a9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:41:31.446385 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-4cf13a20-41e3-4025-b64f-fe75c4798a9b None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:41:31.447689 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-4cf13a20-41e3-4025-b64f-fe75c4798a9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:41:31.507652 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1404/2801] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:41:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:41:31.515996 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea94b183-16e7-4688-8fd3-4373de1843d6 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:41:31.516374 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea94b183-16e7-4688-8fd3-4373de1843d6 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:41:31.516721 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea94b183-16e7-4688-8fd3-4373de1843d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:41:31.535855 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-ea94b183-16e7-4688-8fd3-4373de1843d6 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:41:31.537076 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-ea94b183-16e7-4688-8fd3-4373de1843d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:41:31.544999 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1398/2802] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:41:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 11:41:31.549179 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf13a20-41e3-4025-b64f-fe75c4798a9b None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:41:31.560055 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf13a20-41e3-4025-b64f-fe75c4798a9b None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:41:31.564844 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263d4f78-5bd6-4465-8f5a-bf45532afb93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:41:31.565512 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263d4f78-5bd6-4465-8f5a-bf45532afb93 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:41:31.565798 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263d4f78-5bd6-4465-8f5a-bf45532afb93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:41:31.566078 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263d4f78-5bd6-4465-8f5a-bf45532afb93 admin admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:41:31.566801 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263d4f78-5bd6-4465-8f5a-bf45532afb93 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:41:31.567110 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263d4f78-5bd6-4465-8f5a-bf45532afb93 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:41:31.568748 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263d4f78-5bd6-4465-8f5a-bf45532afb93 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:41:31.580439 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1405/2803] 10.222.0.44 () {58 vars in 1152 bytes} [Wed Jun 29 11:41:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:41:31.585317 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea94b183-16e7-4688-8fd3-4373de1843d6 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:41:31.595258 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea94b183-16e7-4688-8fd3-4373de1843d6 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:41:31.600957 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab5cb3a-858d-4c24-87f6-ef588a63b138 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:41:31.602352 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab5cb3a-858d-4c24-87f6-ef588a63b138 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:41:31.602689 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab5cb3a-858d-4c24-87f6-ef588a63b138 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:41:31.602970 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab5cb3a-858d-4c24-87f6-ef588a63b138 admin admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:41:31.604012 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab5cb3a-858d-4c24-87f6-ef588a63b138 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:41:31.607366 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab5cb3a-858d-4c24-87f6-ef588a63b138 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:41:31.720732 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1399/2804] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jun 29 11:41:31 2022] POST /identity/v3/projects => generated 373 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:41:31.725531 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263d4f78-5bd6-4465-8f5a-bf45532afb93 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:41:31.735883 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263d4f78-5bd6-4465-8f5a-bf45532afb93 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:41:31.794690 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b46f452-95a1-4147-a883-da149b8ee911 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:41:31.795392 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b46f452-95a1-4147-a883-da149b8ee911 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:41:31.795678 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b46f452-95a1-4147-a883-da149b8ee911 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:41:31.795997 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b46f452-95a1-4147-a883-da149b8ee911 admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:41:31.796715 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b46f452-95a1-4147-a883-da149b8ee911 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:41:31.798746 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b46f452-95a1-4147-a883-da149b8ee911 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:41:31.870078 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1406/2805] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jun 29 11:41:31 2022] POST /identity/v3/users => generated 287 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 11:41:31.874347 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab5cb3a-858d-4c24-87f6-ef588a63b138 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:41:31.886664 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab5cb3a-858d-4c24-87f6-ef588a63b138 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:41:31.891424 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b462cf-aaa2-4941-83cd-2ddb348ead62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:41:31.892137 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b462cf-aaa2-4941-83cd-2ddb348ead62 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:41:31.892453 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b462cf-aaa2-4941-83cd-2ddb348ead62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:41:31.892732 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b462cf-aaa2-4941-83cd-2ddb348ead62 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:41:31.893566 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b462cf-aaa2-4941-83cd-2ddb348ead62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:41:31.895825 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b462cf-aaa2-4941-83cd-2ddb348ead62 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:41:31.910229 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1400/2806] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:41:31 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 11:41:31.912229 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b46f452-95a1-4147-a883-da149b8ee911 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:41:31.925345 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b46f452-95a1-4147-a883-da149b8ee911 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:41:31.931273 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633f02c4-cb34-4c2f-9621-83f65e7035dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:41:31.932063 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633f02c4-cb34-4c2f-9621-83f65e7035dd admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:41:31.932653 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633f02c4-cb34-4c2f-9621-83f65e7035dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:41:31.932938 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633f02c4-cb34-4c2f-9621-83f65e7035dd admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:41:31.933736 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633f02c4-cb34-4c2f-9621-83f65e7035dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:41:31.935331 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633f02c4-cb34-4c2f-9621-83f65e7035dd admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:41:31.946294 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1407/2807] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:41:31 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 11:41:31.950573 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b462cf-aaa2-4941-83cd-2ddb348ead62 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:41:31.961138 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b462cf-aaa2-4941-83cd-2ddb348ead62 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:41:31.965732 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7780617a-60b8-4bf6-a914-0ecee17c9bf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:41:31.966598 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7780617a-60b8-4bf6-a914-0ecee17c9bf1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:41:31.967116 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7780617a-60b8-4bf6-a914-0ecee17c9bf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:41:31.967275 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7780617a-60b8-4bf6-a914-0ecee17c9bf1 admin admin] PATH_INFO: `/v3/projects/a4439a41e2c644178f87ef6a77b80fbf/users/5c1e4d67abd44aa2a1c659a1c33babbc/roles/e1c88eebe911432cab6a07fbeb84653b` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:41:31.969787 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:41:31.983615 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7780617a-60b8-4bf6-a914-0ecee17c9bf1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4439a41e2c644178f87ef6a77b80fbf, user_id=5c1e4d67abd44aa2a1c659a1c33babbc, role_id=e1c88eebe911432cab6a07fbeb84653b)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:41:31.986288 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7780617a-60b8-4bf6-a914-0ecee17c9bf1 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:41:32.057989 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1401/2808] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jun 29 11:41:31 2022] PUT /identity/v3/projects/a4439a41e2c644178f87ef6a77b80fbf/users/5c1e4d67abd44aa2a1c659a1c33babbc/roles/e1c88eebe911432cab6a07fbeb84653b => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:41:32.118137 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ff842e-43a8-4fe1-90f3-91f808f7906a None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:41:32.118425 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ff842e-43a8-4fe1-90f3-91f808f7906a None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:41:32.118711 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ff842e-43a8-4fe1-90f3-91f808f7906a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:41:32.139483 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:41:32.148423 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-69ff842e-43a8-4fe1-90f3-91f808f7906a None None] MFA Rules not processed for user `5c1e4d67abd44aa2a1c659a1c33babbc`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:41:32.149745 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-69ff842e-43a8-4fe1-90f3-91f808f7906a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:41:32.217000 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1408/2809] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:41:32 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:41:32.221695 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7780617a-60b8-4bf6-a914-0ecee17c9bf1 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:41:32.287120 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7780617a-60b8-4bf6-a914-0ecee17c9bf1 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:41:32.291925 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bebca2-5a28-4d38-ad95-4deacee9daf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:41:32.292593 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bebca2-5a28-4d38-ad95-4deacee9daf3 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:41:32.292876 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bebca2-5a28-4d38-ad95-4deacee9daf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:41:32.293149 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bebca2-5a28-4d38-ad95-4deacee9daf3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:41:32.293842 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bebca2-5a28-4d38-ad95-4deacee9daf3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:41:32.295764 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bebca2-5a28-4d38-ad95-4deacee9daf3 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:41:32.409154 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1402/2810] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jun 29 11:41:32 2022] POST /identity/v3/projects => generated 371 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:41:32.413714 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ff842e-43a8-4fe1-90f3-91f808f7906a None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:41:32.424659 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ff842e-43a8-4fe1-90f3-91f808f7906a None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:41:32.483662 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de0494e-1513-4a72-9dcf-0e6bed6f3b29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:41:32.484457 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de0494e-1513-4a72-9dcf-0e6bed6f3b29 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:41:32.484801 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de0494e-1513-4a72-9dcf-0e6bed6f3b29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:41:32.485165 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de0494e-1513-4a72-9dcf-0e6bed6f3b29 admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:41:32.485852 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de0494e-1513-4a72-9dcf-0e6bed6f3b29 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:41:32.488458 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de0494e-1513-4a72-9dcf-0e6bed6f3b29 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:41:32.573867 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1409/2811] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jun 29 11:41:32 2022] POST /identity/v3/users => generated 292 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:41:32.578643 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bebca2-5a28-4d38-ad95-4deacee9daf3 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:41:32.589441 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bebca2-5a28-4d38-ad95-4deacee9daf3 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:41:32.594085 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cc617a-be82-4c24-be10-17abd2e20d0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:41:32.594736 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cc617a-be82-4c24-be10-17abd2e20d0d admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:41:32.595022 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cc617a-be82-4c24-be10-17abd2e20d0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:41:32.595302 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cc617a-be82-4c24-be10-17abd2e20d0d admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:41:32.596065 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cc617a-be82-4c24-be10-17abd2e20d0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:41:32.597667 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cc617a-be82-4c24-be10-17abd2e20d0d admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:41:32.609076 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1403/2812] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:41:32 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 11:41:32.614404 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de0494e-1513-4a72-9dcf-0e6bed6f3b29 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:41:32.624540 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de0494e-1513-4a72-9dcf-0e6bed6f3b29 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:41:32.629707 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b3eb78-dfea-4d55-898f-528745365c80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:41:32.630582 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b3eb78-dfea-4d55-898f-528745365c80 admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:41:32.630861 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b3eb78-dfea-4d55-898f-528745365c80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:41:32.631135 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b3eb78-dfea-4d55-898f-528745365c80 admin admin] PATH_INFO: `/v3/projects/050a479e088247ca834c72853d6876cf/users/48252945584649408852500e06b15641/roles/e95c2ce63e114cc19a9dda6ad21e794f` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:41:32.645418 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b3eb78-dfea-4d55-898f-528745365c80 admin admin] RBAC: Authorizing `identity:create_grant(project_id=050a479e088247ca834c72853d6876cf, user_id=48252945584649408852500e06b15641, role_id=e95c2ce63e114cc19a9dda6ad21e794f)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:41:32.648063 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b3eb78-dfea-4d55-898f-528745365c80 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:41:32.715381 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1410/2813] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jun 29 11:41:32 2022] PUT /identity/v3/projects/050a479e088247ca834c72853d6876cf/users/48252945584649408852500e06b15641/roles/e95c2ce63e114cc19a9dda6ad21e794f => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:41:32.719334 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cc617a-be82-4c24-be10-17abd2e20d0d admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:41:32.730011 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cc617a-be82-4c24-be10-17abd2e20d0d admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:41:32.792666 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e59978d-e70a-46e3-b542-919e82980afa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:41:32.793387 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e59978d-e70a-46e3-b542-919e82980afa admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:41:32.793667 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e59978d-e70a-46e3-b542-919e82980afa admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:41:32.793940 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e59978d-e70a-46e3-b542-919e82980afa admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:41:32.794672 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e59978d-e70a-46e3-b542-919e82980afa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:41:32.796358 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e59978d-e70a-46e3-b542-919e82980afa admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:41:32.807231 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1404/2814] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:41:32 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 11:41:32.811524 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b3eb78-dfea-4d55-898f-528745365c80 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:41:32.825515 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b3eb78-dfea-4d55-898f-528745365c80 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:41:32.830386 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d884f0da-3815-45d5-8038-96e0accaff42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:41:32.831308 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d884f0da-3815-45d5-8038-96e0accaff42 admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:41:32.831623 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d884f0da-3815-45d5-8038-96e0accaff42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:41:32.831960 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d884f0da-3815-45d5-8038-96e0accaff42 admin admin] PATH_INFO: `/v3/projects/050a479e088247ca834c72853d6876cf/users/48252945584649408852500e06b15641/roles/e95c2ce63e114cc19a9dda6ad21e794f` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:41:32.833972 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d884f0da-3815-45d5-8038-96e0accaff42 admin admin] RBAC: Authorizing `identity:create_grant(project_id=050a479e088247ca834c72853d6876cf, user_id=48252945584649408852500e06b15641, role_id=e95c2ce63e114cc19a9dda6ad21e794f)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:41:32.836654 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d884f0da-3815-45d5-8038-96e0accaff42 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:41:32.847450 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1411/2815] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jun 29 11:41:32 2022] PUT /identity/v3/projects/050a479e088247ca834c72853d6876cf/users/48252945584649408852500e06b15641/roles/e95c2ce63e114cc19a9dda6ad21e794f => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:41:32.857850 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbeb14a9-efa1-4ef4-97cd-f3a084804f81 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:41:32.858138 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbeb14a9-efa1-4ef4-97cd-f3a084804f81 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:41:32.858419 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbeb14a9-efa1-4ef4-97cd-f3a084804f81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:41:32.885274 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-fbeb14a9-efa1-4ef4-97cd-f3a084804f81 None None] MFA Rules not processed for user `48252945584649408852500e06b15641`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:41:32.886496 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-fbeb14a9-efa1-4ef4-97cd-f3a084804f81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:41:32.930951 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:41:32.960029 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1405/2816] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:41:32 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 11:41:32.971018 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d884f0da-3815-45d5-8038-96e0accaff42 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:41:32.972982 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-d884f0da-3815-45d5-8038-96e0accaff42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:41:33.032890 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d884f0da-3815-45d5-8038-96e0accaff42 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:41:33.037089 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb27e37f-29b9-425e-b7b2-905206ed6c68 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff7931b75a924394b1af733febce0834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:41:33.037733 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb27e37f-29b9-425e-b7b2-905206ed6c68 service cinder] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:41:33.038015 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb27e37f-29b9-425e-b7b2-905206ed6c68 service cinder] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:41:33.038287 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb27e37f-29b9-425e-b7b2-905206ed6c68 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:41:33.044069 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:41:33.047548 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb27e37f-29b9-425e-b7b2-905206ed6c68 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:41:33.048989 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb27e37f-29b9-425e-b7b2-905206ed6c68 service cinder] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:41:33.058758 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1412/2817] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:41:32 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:41:33.995215 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbeb14a9-efa1-4ef4-97cd-f3a084804f81 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:41:34.058524 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbeb14a9-efa1-4ef4-97cd-f3a084804f81 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:41:34.063008 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c2907d-0cbb-4956-81b3-e1f39a369255 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:41:34.064282 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c2907d-0cbb-4956-81b3-e1f39a369255 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:41:34.064282 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c2907d-0cbb-4956-81b3-e1f39a369255 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:41:34.064497 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c2907d-0cbb-4956-81b3-e1f39a369255 admin admin] PATH_INFO: `/v3/users/5c1e4d67abd44aa2a1c659a1c33babbc` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:41:34.065679 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c2907d-0cbb-4956-81b3-e1f39a369255 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c1e4d67abd44aa2a1c659a1c33babbc)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:41:34.067611 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c2907d-0cbb-4956-81b3-e1f39a369255 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:41:34.297075 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-43c2907d-0cbb-4956-81b3-e1f39a369255 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c1e4d67abd44aa2a1c659a1c33babbc'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:41:34.359674 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-43c2907d-0cbb-4956-81b3-e1f39a369255 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c1e4d67abd44aa2a1c659a1c33babbc'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:41:34.360376 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-43c2907d-0cbb-4956-81b3-e1f39a369255 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c1e4d67abd44aa2a1c659a1c33babbc'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:41:34.378592 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-43c2907d-0cbb-4956-81b3-e1f39a369255 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c1e4d67abd44aa2a1c659a1c33babbc'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:41:34.379307 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-43c2907d-0cbb-4956-81b3-e1f39a369255 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c1e4d67abd44aa2a1c659a1c33babbc'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:41:34.392563 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1406/2818] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jun 29 11:41:33 2022] DELETE /identity/v3/users/5c1e4d67abd44aa2a1c659a1c33babbc => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:41:34.397627 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90afb549-ee1f-44d7-bc8a-3f5f53ebed61 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:41:34.398119 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90afb549-ee1f-44d7-bc8a-3f5f53ebed61 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:41:34.398233 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90afb549-ee1f-44d7-bc8a-3f5f53ebed61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:41:34.420864 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-90afb549-ee1f-44d7-bc8a-3f5f53ebed61 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:41:34.422168 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-90afb549-ee1f-44d7-bc8a-3f5f53ebed61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:41:34.482149 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1413/2819] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:41:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:41:34.492452 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c2907d-0cbb-4956-81b3-e1f39a369255 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:41:34.493912 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-43c2907d-0cbb-4956-81b3-e1f39a369255 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:41:34.551485 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c2907d-0cbb-4956-81b3-e1f39a369255 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:41:34.555795 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfd4dcc-8e09-4b8d-9fbb-e3d8abc8276e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f4bb9f989e4024a5da28dced646b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:41:34.556945 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfd4dcc-8e09-4b8d-9fbb-e3d8abc8276e service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:41:34.557231 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfd4dcc-8e09-4b8d-9fbb-e3d8abc8276e service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:41:34.557507 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfd4dcc-8e09-4b8d-9fbb-e3d8abc8276e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:41:34.566785 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfd4dcc-8e09-4b8d-9fbb-e3d8abc8276e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:41:34.568256 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfd4dcc-8e09-4b8d-9fbb-e3d8abc8276e service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:41:34.573171 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:41:34.577996 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1407/2820] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:41:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:41:35.002007 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90afb549-ee1f-44d7-bc8a-3f5f53ebed61 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:41:35.003494 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-90afb549-ee1f-44d7-bc8a-3f5f53ebed61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:41:35.015556 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90afb549-ee1f-44d7-bc8a-3f5f53ebed61 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:41:35.020294 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bac4d7b-41f5-4406-b8cf-ebcd26c1a5fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:41:35.021156 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bac4d7b-41f5-4406-b8cf-ebcd26c1a5fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:41:35.021441 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bac4d7b-41f5-4406-b8cf-ebcd26c1a5fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:41:35.021719 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bac4d7b-41f5-4406-b8cf-ebcd26c1a5fd admin admin] PATH_INFO: `/v3/projects/a4439a41e2c644178f87ef6a77b80fbf` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:41:35.022959 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bac4d7b-41f5-4406-b8cf-ebcd26c1a5fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=a4439a41e2c644178f87ef6a77b80fbf)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:41:35.025127 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bac4d7b-41f5-4406-b8cf-ebcd26c1a5fd admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:41:35.131570 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-1bac4d7b-41f5-4406-b8cf-ebcd26c1a5fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a4439a41e2c644178f87ef6a77b80fbf', 'request_id': 'req-1bac4d7b-41f5-4406-b8cf-ebcd26c1a5fd'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:41:35.137887 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-1bac4d7b-41f5-4406-b8cf-ebcd26c1a5fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a4439a41e2c644178f87ef6a77b80fbf', 'request_id': 'req-1bac4d7b-41f5-4406-b8cf-ebcd26c1a5fd'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:41:35.143603 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-1bac4d7b-41f5-4406-b8cf-ebcd26c1a5fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a4439a41e2c644178f87ef6a77b80fbf', 'request_id': 'req-1bac4d7b-41f5-4406-b8cf-ebcd26c1a5fd'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:41:35.205223 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-1bac4d7b-41f5-4406-b8cf-ebcd26c1a5fd admin admin] The token cache is being invalidate because project a4439a41e2c644178f87ef6a77b80fbf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 11:41:35.205512 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-1bac4d7b-41f5-4406-b8cf-ebcd26c1a5fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:41:35.206137 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-1bac4d7b-41f5-4406-b8cf-ebcd26c1a5fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:41:35.207676 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1414/2821] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jun 29 11:41:35 2022] DELETE /identity/v3/projects/a4439a41e2c644178f87ef6a77b80fbf => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:41:35.211599 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfd4dcc-8e09-4b8d-9fbb-e3d8abc8276e service neutron] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:41:35.221270 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfd4dcc-8e09-4b8d-9fbb-e3d8abc8276e service neutron] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:41:35.277357 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb2dc30-f898-452d-9237-bed3d555cd4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:41:35.278148 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb2dc30-f898-452d-9237-bed3d555cd4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:41:35.278434 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb2dc30-f898-452d-9237-bed3d555cd4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:41:35.278716 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb2dc30-f898-452d-9237-bed3d555cd4a admin admin] PATH_INFO: `/v3/users/48252945584649408852500e06b15641` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:41:35.279895 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb2dc30-f898-452d-9237-bed3d555cd4a admin admin] RBAC: Authorizing `identity:delete_user(user_id=48252945584649408852500e06b15641)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:41:35.281796 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb2dc30-f898-452d-9237-bed3d555cd4a admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:41:35.436881 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-5fb2dc30-f898-452d-9237-bed3d555cd4a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '48252945584649408852500e06b15641'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:41:35.497648 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-5fb2dc30-f898-452d-9237-bed3d555cd4a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '48252945584649408852500e06b15641'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:41:35.498270 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-5fb2dc30-f898-452d-9237-bed3d555cd4a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '48252945584649408852500e06b15641'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:41:35.510481 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-5fb2dc30-f898-452d-9237-bed3d555cd4a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '48252945584649408852500e06b15641'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:41:35.511118 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-5fb2dc30-f898-452d-9237-bed3d555cd4a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '48252945584649408852500e06b15641'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:41:35.523659 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1408/2822] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jun 29 11:41:35 2022] DELETE /identity/v3/users/48252945584649408852500e06b15641 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:41:35.915246 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bac4d7b-41f5-4406-b8cf-ebcd26c1a5fd admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:41:35.918888 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-1bac4d7b-41f5-4406-b8cf-ebcd26c1a5fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:41:35.937185 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bac4d7b-41f5-4406-b8cf-ebcd26c1a5fd admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:41:35.996388 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521caa14-b574-459e-b876-f22066967c4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:41:35.997246 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521caa14-b574-459e-b876-f22066967c4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:41:35.997527 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521caa14-b574-459e-b876-f22066967c4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:41:35.997805 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521caa14-b574-459e-b876-f22066967c4e admin admin] PATH_INFO: `/v3/projects/050a479e088247ca834c72853d6876cf` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:41:35.998968 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521caa14-b574-459e-b876-f22066967c4e admin admin] RBAC: Authorizing `identity:delete_project(project_id=050a479e088247ca834c72853d6876cf)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:41:36.000894 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521caa14-b574-459e-b876-f22066967c4e admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:41:36.022606 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:41:36.146596 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-521caa14-b574-459e-b876-f22066967c4e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '050a479e088247ca834c72853d6876cf', 'request_id': 'req-521caa14-b574-459e-b876-f22066967c4e'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:41:36.152925 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-521caa14-b574-459e-b876-f22066967c4e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '050a479e088247ca834c72853d6876cf', 'request_id': 'req-521caa14-b574-459e-b876-f22066967c4e'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:41:36.159400 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-521caa14-b574-459e-b876-f22066967c4e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '050a479e088247ca834c72853d6876cf', 'request_id': 'req-521caa14-b574-459e-b876-f22066967c4e'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:41:36.274935 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-521caa14-b574-459e-b876-f22066967c4e admin admin] The token cache is being invalidate because project 050a479e088247ca834c72853d6876cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 11:41:36.275225 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-521caa14-b574-459e-b876-f22066967c4e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:41:36.275864 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-521caa14-b574-459e-b876-f22066967c4e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:41:36.277394 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1415/2823] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jun 29 11:41:35 2022] DELETE /identity/v3/projects/050a479e088247ca834c72853d6876cf => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:41:36.282253 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0e9c93-50bd-4d21-a8ec-5e7ddb529264 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:41:36.288281 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0e9c93-50bd-4d21-a8ec-5e7ddb529264 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:41:36.288281 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0e9c93-50bd-4d21-a8ec-5e7ddb529264 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:41:36.301093 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-5b0e9c93-50bd-4d21-a8ec-5e7ddb529264 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:41:36.302298 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-5b0e9c93-50bd-4d21-a8ec-5e7ddb529264 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:41:36.362823 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1409/2824] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:41:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:41:36.370729 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029a34ef-208d-4ff1-9b41-0737599ece25 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:41:36.371019 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029a34ef-208d-4ff1-9b41-0737599ece25 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:41:36.371304 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029a34ef-208d-4ff1-9b41-0737599ece25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:41:36.390443 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-029a34ef-208d-4ff1-9b41-0737599ece25 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:41:36.391650 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-029a34ef-208d-4ff1-9b41-0737599ece25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:41:36.399609 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1416/2825] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:41:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:41:36.403980 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0e9c93-50bd-4d21-a8ec-5e7ddb529264 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:41:36.414128 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0e9c93-50bd-4d21-a8ec-5e7ddb529264 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:41:36.418701 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f88da41-457d-45b2-9ae1-8ec2a2a92cb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:41:36.419351 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f88da41-457d-45b2-9ae1-8ec2a2a92cb1 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:41:36.419635 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f88da41-457d-45b2-9ae1-8ec2a2a92cb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:41:36.419949 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f88da41-457d-45b2-9ae1-8ec2a2a92cb1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:41:36.420680 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f88da41-457d-45b2-9ae1-8ec2a2a92cb1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:41:36.420988 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f88da41-457d-45b2-9ae1-8ec2a2a92cb1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:41:36.422578 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f88da41-457d-45b2-9ae1-8ec2a2a92cb1 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:41:36.434574 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1410/2826] 10.222.0.44 () {58 vars in 1152 bytes} [Wed Jun 29 11:41:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:41:36.438583 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029a34ef-208d-4ff1-9b41-0737599ece25 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:41:36.447915 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029a34ef-208d-4ff1-9b41-0737599ece25 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:41:36.452605 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93979d3b-3f0b-44aa-9ef0-09258357bb3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:41:36.453257 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93979d3b-3f0b-44aa-9ef0-09258357bb3c admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:41:36.453543 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93979d3b-3f0b-44aa-9ef0-09258357bb3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:41:36.453821 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93979d3b-3f0b-44aa-9ef0-09258357bb3c admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:41:36.454572 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93979d3b-3f0b-44aa-9ef0-09258357bb3c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:41:36.456510 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93979d3b-3f0b-44aa-9ef0-09258357bb3c admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:41:36.586524 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1417/2827] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jun 29 11:41:36 2022] POST /identity/v3/projects => generated 369 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:41:36.590615 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f88da41-457d-45b2-9ae1-8ec2a2a92cb1 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:41:36.601403 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f88da41-457d-45b2-9ae1-8ec2a2a92cb1 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:41:36.605047 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:41:36.656310 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7eba27-625a-4c9e-becb-242f6cad29ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:41:36.656904 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7eba27-625a-4c9e-becb-242f6cad29ea admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:41:36.657381 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7eba27-625a-4c9e-becb-242f6cad29ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:41:36.657822 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7eba27-625a-4c9e-becb-242f6cad29ea admin admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:41:36.658875 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7eba27-625a-4c9e-becb-242f6cad29ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:41:36.661325 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7eba27-625a-4c9e-becb-242f6cad29ea admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:41:36.766779 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1411/2828] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jun 29 11:41:36 2022] POST /identity/v3/users => generated 285 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 11:41:36.771358 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93979d3b-3f0b-44aa-9ef0-09258357bb3c admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:41:36.781742 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93979d3b-3f0b-44aa-9ef0-09258357bb3c admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:41:36.787060 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a4ad0a-894a-4ec7-b8a0-74ff85312f61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:41:36.787712 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a4ad0a-894a-4ec7-b8a0-74ff85312f61 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:41:36.788027 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a4ad0a-894a-4ec7-b8a0-74ff85312f61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:41:36.788307 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a4ad0a-894a-4ec7-b8a0-74ff85312f61 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:41:36.789026 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a4ad0a-894a-4ec7-b8a0-74ff85312f61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:41:36.790606 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a4ad0a-894a-4ec7-b8a0-74ff85312f61 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:41:36.801776 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1418/2829] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:41:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 11:41:36.805449 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7eba27-625a-4c9e-becb-242f6cad29ea admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:41:36.815259 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7eba27-625a-4c9e-becb-242f6cad29ea admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:41:36.819790 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c968febb-4fb8-44f4-b671-cc0d53fc0a3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:41:36.820462 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c968febb-4fb8-44f4-b671-cc0d53fc0a3d admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:41:36.820747 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c968febb-4fb8-44f4-b671-cc0d53fc0a3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:41:36.821024 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c968febb-4fb8-44f4-b671-cc0d53fc0a3d admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:41:36.821746 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c968febb-4fb8-44f4-b671-cc0d53fc0a3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:41:36.823330 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c968febb-4fb8-44f4-b671-cc0d53fc0a3d admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:41:36.833670 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1412/2830] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:41:36 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 11:41:36.837504 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a4ad0a-894a-4ec7-b8a0-74ff85312f61 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:41:36.847723 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a4ad0a-894a-4ec7-b8a0-74ff85312f61 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:41:36.852955 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f7d0b8-84d3-44b0-8d23-01d88613825a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:41:36.853831 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f7d0b8-84d3-44b0-8d23-01d88613825a admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:41:36.854115 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f7d0b8-84d3-44b0-8d23-01d88613825a admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:41:36.854389 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f7d0b8-84d3-44b0-8d23-01d88613825a admin admin] PATH_INFO: `/v3/projects/86b2ed977d994ffa89b5721f95b91797/users/1483cc14be7341db955776bbd9a76605/roles/e1c88eebe911432cab6a07fbeb84653b` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:41:36.872576 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f7d0b8-84d3-44b0-8d23-01d88613825a admin admin] RBAC: Authorizing `identity:create_grant(project_id=86b2ed977d994ffa89b5721f95b91797, user_id=1483cc14be7341db955776bbd9a76605, role_id=e1c88eebe911432cab6a07fbeb84653b)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:41:36.875249 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f7d0b8-84d3-44b0-8d23-01d88613825a admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:41:36.943029 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1419/2831] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jun 29 11:41:36 2022] PUT /identity/v3/projects/86b2ed977d994ffa89b5721f95b91797/users/1483cc14be7341db955776bbd9a76605/roles/e1c88eebe911432cab6a07fbeb84653b => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:41:36.954421 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58747d0-bbbe-414d-8249-0bb16cf87180 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:41:36.954742 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58747d0-bbbe-414d-8249-0bb16cf87180 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:41:36.955029 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58747d0-bbbe-414d-8249-0bb16cf87180 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:41:36.981991 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-a58747d0-bbbe-414d-8249-0bb16cf87180 None None] MFA Rules not processed for user `1483cc14be7341db955776bbd9a76605`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:41:36.983214 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-a58747d0-bbbe-414d-8249-0bb16cf87180 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:41:37.046786 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1413/2832] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:41:36 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:41:37.051244 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f7d0b8-84d3-44b0-8d23-01d88613825a admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:41:37.111737 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f7d0b8-84d3-44b0-8d23-01d88613825a admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:41:37.116725 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956923fc-ef1e-4d24-ac1e-10920f79bfe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:41:37.117375 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956923fc-ef1e-4d24-ac1e-10920f79bfe6 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:41:37.117662 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956923fc-ef1e-4d24-ac1e-10920f79bfe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:41:37.117935 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956923fc-ef1e-4d24-ac1e-10920f79bfe6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:41:37.118635 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956923fc-ef1e-4d24-ac1e-10920f79bfe6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:41:37.120578 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956923fc-ef1e-4d24-ac1e-10920f79bfe6 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:41:37.207093 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1420/2833] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jun 29 11:41:37 2022] POST /identity/v3/projects => generated 367 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 11:41:37.210929 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58747d0-bbbe-414d-8249-0bb16cf87180 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:41:37.222845 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58747d0-bbbe-414d-8249-0bb16cf87180 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:41:37.277412 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c42b49-c6e7-42df-9f2d-5b0539cc9529 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:41:37.278071 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c42b49-c6e7-42df-9f2d-5b0539cc9529 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:41:37.278353 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c42b49-c6e7-42df-9f2d-5b0539cc9529 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:41:37.278628 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c42b49-c6e7-42df-9f2d-5b0539cc9529 admin admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:41:37.279312 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c42b49-c6e7-42df-9f2d-5b0539cc9529 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:41:37.281253 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c42b49-c6e7-42df-9f2d-5b0539cc9529 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:41:37.360424 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1414/2834] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jun 29 11:41:37 2022] POST /identity/v3/users => generated 290 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:41:37.364979 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956923fc-ef1e-4d24-ac1e-10920f79bfe6 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:41:37.375341 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956923fc-ef1e-4d24-ac1e-10920f79bfe6 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:41:37.380032 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6a8a23-c369-4615-abd8-8b7ce7eb544d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:41:37.380710 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6a8a23-c369-4615-abd8-8b7ce7eb544d admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:41:37.380997 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6a8a23-c369-4615-abd8-8b7ce7eb544d admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:41:37.381305 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6a8a23-c369-4615-abd8-8b7ce7eb544d admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:41:37.382107 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6a8a23-c369-4615-abd8-8b7ce7eb544d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:41:37.383658 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6a8a23-c369-4615-abd8-8b7ce7eb544d admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:41:37.394212 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1421/2835] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:41:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 11:41:37.398116 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c42b49-c6e7-42df-9f2d-5b0539cc9529 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:41:37.404382 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:41:37.408020 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c42b49-c6e7-42df-9f2d-5b0539cc9529 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:41:37.413270 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffd8756-63e4-4dbf-9c9b-66c8358fe5d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:41:37.415923 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffd8756-63e4-4dbf-9c9b-66c8358fe5d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:41:37.415923 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffd8756-63e4-4dbf-9c9b-66c8358fe5d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:41:37.415923 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffd8756-63e4-4dbf-9c9b-66c8358fe5d2 admin admin] PATH_INFO: `/v3/projects/3108d2c07e674fe69907683c9e19cef0/users/0da08f771e71434aa467937699cda494/roles/e95c2ce63e114cc19a9dda6ad21e794f` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:41:37.429933 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffd8756-63e4-4dbf-9c9b-66c8358fe5d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3108d2c07e674fe69907683c9e19cef0, user_id=0da08f771e71434aa467937699cda494, role_id=e95c2ce63e114cc19a9dda6ad21e794f)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:41:37.432662 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffd8756-63e4-4dbf-9c9b-66c8358fe5d2 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:41:37.559091 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1415/2836] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jun 29 11:41:37 2022] PUT /identity/v3/projects/3108d2c07e674fe69907683c9e19cef0/users/0da08f771e71434aa467937699cda494/roles/e95c2ce63e114cc19a9dda6ad21e794f => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:41:37.563313 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6a8a23-c369-4615-abd8-8b7ce7eb544d admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:41:37.574130 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6a8a23-c369-4615-abd8-8b7ce7eb544d admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:41:37.586693 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:41:37.630480 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9181ebe1-1bda-47c2-a384-326f454fd046 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:41:37.631188 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9181ebe1-1bda-47c2-a384-326f454fd046 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:41:37.631477 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9181ebe1-1bda-47c2-a384-326f454fd046 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:41:37.631755 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9181ebe1-1bda-47c2-a384-326f454fd046 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:41:37.632538 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9181ebe1-1bda-47c2-a384-326f454fd046 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:41:37.634167 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9181ebe1-1bda-47c2-a384-326f454fd046 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:41:37.643723 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1422/2837] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:41:37 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 11:41:37.647817 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffd8756-63e4-4dbf-9c9b-66c8358fe5d2 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:41:37.658033 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffd8756-63e4-4dbf-9c9b-66c8358fe5d2 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:41:37.662887 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8241efe0-2ebd-476d-a10c-c1bcfdfb5d77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:41:37.663756 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8241efe0-2ebd-476d-a10c-c1bcfdfb5d77 admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:41:37.664065 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8241efe0-2ebd-476d-a10c-c1bcfdfb5d77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:41:37.664342 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8241efe0-2ebd-476d-a10c-c1bcfdfb5d77 admin admin] PATH_INFO: `/v3/projects/3108d2c07e674fe69907683c9e19cef0/users/0da08f771e71434aa467937699cda494/roles/e95c2ce63e114cc19a9dda6ad21e794f` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:41:37.666473 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8241efe0-2ebd-476d-a10c-c1bcfdfb5d77 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3108d2c07e674fe69907683c9e19cef0, user_id=0da08f771e71434aa467937699cda494, role_id=e95c2ce63e114cc19a9dda6ad21e794f)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:41:37.669236 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8241efe0-2ebd-476d-a10c-c1bcfdfb5d77 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:41:37.680094 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1416/2838] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jun 29 11:41:37 2022] PUT /identity/v3/projects/3108d2c07e674fe69907683c9e19cef0/users/0da08f771e71434aa467937699cda494/roles/e95c2ce63e114cc19a9dda6ad21e794f => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:41:37.690056 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a452ba-70fa-473a-8d29-72b4819fbee0 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:41:37.690879 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a452ba-70fa-473a-8d29-72b4819fbee0 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:41:37.691161 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a452ba-70fa-473a-8d29-72b4819fbee0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:41:37.716581 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-69a452ba-70fa-473a-8d29-72b4819fbee0 None None] MFA Rules not processed for user `0da08f771e71434aa467937699cda494`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:41:37.717784 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-69a452ba-70fa-473a-8d29-72b4819fbee0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:41:37.783702 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1423/2839] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:41:37 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:41:37.794727 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8241efe0-2ebd-476d-a10c-c1bcfdfb5d77 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:41:37.796467 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-8241efe0-2ebd-476d-a10c-c1bcfdfb5d77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:41:37.853503 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8241efe0-2ebd-476d-a10c-c1bcfdfb5d77 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:41:37.857243 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6762f4-2c9f-4e86-b549-bd237503305f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff7931b75a924394b1af733febce0834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:41:37.857888 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6762f4-2c9f-4e86-b549-bd237503305f service cinder] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:41:37.858172 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6762f4-2c9f-4e86-b549-bd237503305f service cinder] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:41:37.858449 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6762f4-2c9f-4e86-b549-bd237503305f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:41:37.867648 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6762f4-2c9f-4e86-b549-bd237503305f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:41:37.869098 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6762f4-2c9f-4e86-b549-bd237503305f service cinder] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:41:37.878891 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1417/2840] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:41:37 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:41:41.136898 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a452ba-70fa-473a-8d29-72b4819fbee0 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:41:41.197434 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a452ba-70fa-473a-8d29-72b4819fbee0 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:41:41.201911 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b343d505-d5df-4149-91dc-d2dc1d9f50c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:41:41.202774 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b343d505-d5df-4149-91dc-d2dc1d9f50c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:41:41.203063 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b343d505-d5df-4149-91dc-d2dc1d9f50c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:41:41.203341 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b343d505-d5df-4149-91dc-d2dc1d9f50c5 admin admin] PATH_INFO: `/v3/users/1483cc14be7341db955776bbd9a76605` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:41:41.204539 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b343d505-d5df-4149-91dc-d2dc1d9f50c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1483cc14be7341db955776bbd9a76605)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:41:41.207027 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b343d505-d5df-4149-91dc-d2dc1d9f50c5 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:41:41.390953 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:41:41.395820 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-b343d505-d5df-4149-91dc-d2dc1d9f50c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1483cc14be7341db955776bbd9a76605'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:41:41.396501 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-b343d505-d5df-4149-91dc-d2dc1d9f50c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1483cc14be7341db955776bbd9a76605'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:41:41.507757 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-b343d505-d5df-4149-91dc-d2dc1d9f50c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1483cc14be7341db955776bbd9a76605'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:41:41.515009 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-b343d505-d5df-4149-91dc-d2dc1d9f50c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1483cc14be7341db955776bbd9a76605'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:41:41.515639 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-b343d505-d5df-4149-91dc-d2dc1d9f50c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1483cc14be7341db955776bbd9a76605'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:41:41.532617 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1424/2841] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jun 29 11:41:41 2022] DELETE /identity/v3/users/1483cc14be7341db955776bbd9a76605 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:41:41.537065 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a59326d-5b10-432b-a1eb-ac5e4297749d None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:41:41.537543 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a59326d-5b10-432b-a1eb-ac5e4297749d None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:41:41.537647 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a59326d-5b10-432b-a1eb-ac5e4297749d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:41:41.555390 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-9a59326d-5b10-432b-a1eb-ac5e4297749d None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:41:41.556643 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-9a59326d-5b10-432b-a1eb-ac5e4297749d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:41:41.619684 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1418/2842] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:41:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:41:41.629944 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b343d505-d5df-4149-91dc-d2dc1d9f50c5 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:41:41.631355 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-b343d505-d5df-4149-91dc-d2dc1d9f50c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:41:41.689121 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b343d505-d5df-4149-91dc-d2dc1d9f50c5 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:41:41.693021 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd5acb9-db7c-447b-b28d-8850b7aa5731 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f4bb9f989e4024a5da28dced646b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:41:41.693670 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd5acb9-db7c-447b-b28d-8850b7aa5731 service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:41:41.693952 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd5acb9-db7c-447b-b28d-8850b7aa5731 service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:41:41.694229 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd5acb9-db7c-447b-b28d-8850b7aa5731 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:41:41.703234 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd5acb9-db7c-447b-b28d-8850b7aa5731 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:41:41.704685 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd5acb9-db7c-447b-b28d-8850b7aa5731 service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:41:41.714031 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1425/2843] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:41:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:41:42.088450 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a59326d-5b10-432b-a1eb-ac5e4297749d None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:41:42.090969 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-9a59326d-5b10-432b-a1eb-ac5e4297749d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:41:42.103628 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a59326d-5b10-432b-a1eb-ac5e4297749d None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:41:42.108287 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48edc56-4dfb-4197-ab04-1ec243ab3942 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:41:42.109184 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48edc56-4dfb-4197-ab04-1ec243ab3942 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:41:42.109470 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48edc56-4dfb-4197-ab04-1ec243ab3942 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:41:42.109749 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48edc56-4dfb-4197-ab04-1ec243ab3942 admin admin] PATH_INFO: `/v3/projects/86b2ed977d994ffa89b5721f95b91797` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:41:42.111263 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48edc56-4dfb-4197-ab04-1ec243ab3942 admin admin] RBAC: Authorizing `identity:delete_project(project_id=86b2ed977d994ffa89b5721f95b91797)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:41:42.113570 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48edc56-4dfb-4197-ab04-1ec243ab3942 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:41:42.235423 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-d48edc56-4dfb-4197-ab04-1ec243ab3942 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '86b2ed977d994ffa89b5721f95b91797', 'request_id': 'req-d48edc56-4dfb-4197-ab04-1ec243ab3942'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:41:42.331556 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-d48edc56-4dfb-4197-ab04-1ec243ab3942 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '86b2ed977d994ffa89b5721f95b91797', 'request_id': 'req-d48edc56-4dfb-4197-ab04-1ec243ab3942'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:41:42.338420 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-d48edc56-4dfb-4197-ab04-1ec243ab3942 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '86b2ed977d994ffa89b5721f95b91797', 'request_id': 'req-d48edc56-4dfb-4197-ab04-1ec243ab3942'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:41:42.342386 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:41:42.350506 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-d48edc56-4dfb-4197-ab04-1ec243ab3942 admin admin] The token cache is being invalidate because project 86b2ed977d994ffa89b5721f95b91797 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 11:41:42.350794 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-d48edc56-4dfb-4197-ab04-1ec243ab3942 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:41:42.351423 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-d48edc56-4dfb-4197-ab04-1ec243ab3942 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:41:42.352986 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1419/2844] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jun 29 11:41:42 2022] DELETE /identity/v3/projects/86b2ed977d994ffa89b5721f95b91797 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:41:42.357251 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd5acb9-db7c-447b-b28d-8850b7aa5731 service neutron] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:41:42.366989 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd5acb9-db7c-447b-b28d-8850b7aa5731 service neutron] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:41:42.423598 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af72aa6e-f7a3-4be1-8966-c05701b8b35c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:41:42.426009 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af72aa6e-f7a3-4be1-8966-c05701b8b35c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:41:42.426009 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af72aa6e-f7a3-4be1-8966-c05701b8b35c admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:41:42.426009 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af72aa6e-f7a3-4be1-8966-c05701b8b35c admin admin] PATH_INFO: `/v3/users/0da08f771e71434aa467937699cda494` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:41:42.426323 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af72aa6e-f7a3-4be1-8966-c05701b8b35c admin admin] RBAC: Authorizing `identity:delete_user(user_id=0da08f771e71434aa467937699cda494)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:41:42.428217 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af72aa6e-f7a3-4be1-8966-c05701b8b35c admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:41:42.653937 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-af72aa6e-f7a3-4be1-8966-c05701b8b35c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0da08f771e71434aa467937699cda494'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:41:42.654534 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-af72aa6e-f7a3-4be1-8966-c05701b8b35c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0da08f771e71434aa467937699cda494'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:41:42.738706 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-af72aa6e-f7a3-4be1-8966-c05701b8b35c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0da08f771e71434aa467937699cda494'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:41:42.746238 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-af72aa6e-f7a3-4be1-8966-c05701b8b35c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0da08f771e71434aa467937699cda494'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:41:42.746926 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-af72aa6e-f7a3-4be1-8966-c05701b8b35c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0da08f771e71434aa467937699cda494'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:41:42.748890 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:41:42.765012 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1426/2845] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jun 29 11:41:42 2022] DELETE /identity/v3/users/0da08f771e71434aa467937699cda494 => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:41:43.217766 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48edc56-4dfb-4197-ab04-1ec243ab3942 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:41:43.230619 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-d48edc56-4dfb-4197-ab04-1ec243ab3942 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:41:43.234774 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48edc56-4dfb-4197-ab04-1ec243ab3942 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:41:43.291299 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a43e16a-bef1-4c57-bad1-94195119940c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:41:43.292193 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a43e16a-bef1-4c57-bad1-94195119940c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:41:43.292481 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a43e16a-bef1-4c57-bad1-94195119940c admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:41:43.292759 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a43e16a-bef1-4c57-bad1-94195119940c admin admin] PATH_INFO: `/v3/projects/3108d2c07e674fe69907683c9e19cef0` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:41:43.293944 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a43e16a-bef1-4c57-bad1-94195119940c admin admin] RBAC: Authorizing `identity:delete_project(project_id=3108d2c07e674fe69907683c9e19cef0)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:41:43.295859 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a43e16a-bef1-4c57-bad1-94195119940c admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:41:43.435348 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-0a43e16a-bef1-4c57-bad1-94195119940c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3108d2c07e674fe69907683c9e19cef0', 'request_id': 'req-0a43e16a-bef1-4c57-bad1-94195119940c'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:41:43.514705 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-0a43e16a-bef1-4c57-bad1-94195119940c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3108d2c07e674fe69907683c9e19cef0', 'request_id': 'req-0a43e16a-bef1-4c57-bad1-94195119940c'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:41:43.522001 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-0a43e16a-bef1-4c57-bad1-94195119940c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3108d2c07e674fe69907683c9e19cef0', 'request_id': 'req-0a43e16a-bef1-4c57-bad1-94195119940c'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:41:43.532214 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-0a43e16a-bef1-4c57-bad1-94195119940c admin admin] The token cache is being invalidate because project 3108d2c07e674fe69907683c9e19cef0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 11:41:43.532478 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-0a43e16a-bef1-4c57-bad1-94195119940c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:41:43.533098 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-0a43e16a-bef1-4c57-bad1-94195119940c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:41:43.535175 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1420/2846] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jun 29 11:41:43 2022] DELETE /identity/v3/projects/3108d2c07e674fe69907683c9e19cef0 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:41:43.540192 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd46a035-f2ca-48c4-8c7d-cead6c263104 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:41:43.540687 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd46a035-f2ca-48c4-8c7d-cead6c263104 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:41:43.540791 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd46a035-f2ca-48c4-8c7d-cead6c263104 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:41:43.559690 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-cd46a035-f2ca-48c4-8c7d-cead6c263104 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:41:43.560924 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-cd46a035-f2ca-48c4-8c7d-cead6c263104 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:41:43.618724 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1427/2847] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:41:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:41:43.637115 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ac81be-5035-4f52-a9bb-2a4c45440630 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:41:43.637407 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ac81be-5035-4f52-a9bb-2a4c45440630 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:41:43.637687 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ac81be-5035-4f52-a9bb-2a4c45440630 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:41:43.656275 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-e4ac81be-5035-4f52-a9bb-2a4c45440630 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:41:43.657471 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-e4ac81be-5035-4f52-a9bb-2a4c45440630 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:41:43.665952 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1421/2848] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:41:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:41:43.670155 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd46a035-f2ca-48c4-8c7d-cead6c263104 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:41:43.679853 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd46a035-f2ca-48c4-8c7d-cead6c263104 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:41:43.684324 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96707e0d-b23a-4504-9981-460e2e9f0d7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:41:43.685243 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96707e0d-b23a-4504-9981-460e2e9f0d7e admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:41:43.685522 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96707e0d-b23a-4504-9981-460e2e9f0d7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:41:43.685791 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96707e0d-b23a-4504-9981-460e2e9f0d7e admin admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:41:43.686531 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96707e0d-b23a-4504-9981-460e2e9f0d7e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:41:43.686831 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96707e0d-b23a-4504-9981-460e2e9f0d7e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:41:43.688627 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96707e0d-b23a-4504-9981-460e2e9f0d7e admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:41:43.704974 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1428/2849] 10.222.0.44 () {58 vars in 1152 bytes} [Wed Jun 29 11:41:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:41:43.709674 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ac81be-5035-4f52-a9bb-2a4c45440630 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:41:43.718944 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ac81be-5035-4f52-a9bb-2a4c45440630 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:41:43.724116 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0733cfc7-4db1-4292-9028-17f89c39fe35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:41:43.724765 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0733cfc7-4db1-4292-9028-17f89c39fe35 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:41:43.725052 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0733cfc7-4db1-4292-9028-17f89c39fe35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:41:43.725331 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0733cfc7-4db1-4292-9028-17f89c39fe35 admin admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:41:43.726031 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0733cfc7-4db1-4292-9028-17f89c39fe35 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:41:43.727957 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0733cfc7-4db1-4292-9028-17f89c39fe35 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:41:43.903879 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1422/2850] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jun 29 11:41:43 2022] POST /identity/v3/projects => generated 385 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:41:43.907910 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96707e0d-b23a-4504-9981-460e2e9f0d7e admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:41:43.918005 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96707e0d-b23a-4504-9981-460e2e9f0d7e admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:41:43.973953 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22464d89-ab38-41ed-af5a-2e82b3013605 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:41:43.974548 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22464d89-ab38-41ed-af5a-2e82b3013605 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:41:43.974833 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22464d89-ab38-41ed-af5a-2e82b3013605 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:41:43.975109 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22464d89-ab38-41ed-af5a-2e82b3013605 admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:41:43.975804 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22464d89-ab38-41ed-af5a-2e82b3013605 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:41:43.977799 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22464d89-ab38-41ed-af5a-2e82b3013605 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:41:44.090810 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1429/2851] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jun 29 11:41:43 2022] POST /identity/v3/users => generated 293 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:41:44.094820 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0733cfc7-4db1-4292-9028-17f89c39fe35 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:41:44.105264 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0733cfc7-4db1-4292-9028-17f89c39fe35 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:41:44.109893 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d02c9c-2196-4e7b-a3a0-dd5e3fb6384c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:41:44.116428 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d02c9c-2196-4e7b-a3a0-dd5e3fb6384c admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:41:44.116428 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d02c9c-2196-4e7b-a3a0-dd5e3fb6384c admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:41:44.116428 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d02c9c-2196-4e7b-a3a0-dd5e3fb6384c admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:41:44.116428 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d02c9c-2196-4e7b-a3a0-dd5e3fb6384c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:41:44.116428 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d02c9c-2196-4e7b-a3a0-dd5e3fb6384c admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:41:44.124189 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1423/2852] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:41:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 11:41:44.128372 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22464d89-ab38-41ed-af5a-2e82b3013605 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:41:44.138438 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22464d89-ab38-41ed-af5a-2e82b3013605 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:41:44.142969 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2ffe5e-0221-463b-a0dc-cd7323b17751 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:41:44.144218 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2ffe5e-0221-463b-a0dc-cd7323b17751 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:41:44.144507 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2ffe5e-0221-463b-a0dc-cd7323b17751 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:41:44.144785 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2ffe5e-0221-463b-a0dc-cd7323b17751 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:41:44.145502 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2ffe5e-0221-463b-a0dc-cd7323b17751 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:41:44.147188 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2ffe5e-0221-463b-a0dc-cd7323b17751 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:41:44.157039 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1430/2853] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:41:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 11:41:44.160917 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d02c9c-2196-4e7b-a3a0-dd5e3fb6384c admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:41:44.170562 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d02c9c-2196-4e7b-a3a0-dd5e3fb6384c admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:41:44.175005 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a35a9b1-c5c2-4f43-8a03-7ae0a4018a99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:41:44.175896 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a35a9b1-c5c2-4f43-8a03-7ae0a4018a99 admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:41:44.176180 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a35a9b1-c5c2-4f43-8a03-7ae0a4018a99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:41:44.176456 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a35a9b1-c5c2-4f43-8a03-7ae0a4018a99 admin admin] PATH_INFO: `/v3/projects/8107c80665bd4171a81f22edb640a432/users/ea0fee783782439ebc8bcd0b957b6459/roles/e1c88eebe911432cab6a07fbeb84653b` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:41:44.191168 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a35a9b1-c5c2-4f43-8a03-7ae0a4018a99 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8107c80665bd4171a81f22edb640a432, user_id=ea0fee783782439ebc8bcd0b957b6459, role_id=e1c88eebe911432cab6a07fbeb84653b)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:41:44.193810 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a35a9b1-c5c2-4f43-8a03-7ae0a4018a99 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:41:44.303702 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1424/2854] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jun 29 11:41:44 2022] PUT /identity/v3/projects/8107c80665bd4171a81f22edb640a432/users/ea0fee783782439ebc8bcd0b957b6459/roles/e1c88eebe911432cab6a07fbeb84653b => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:41:44.313800 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e817fc-80d7-4d27-b4ac-055953957130 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:41:44.314095 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e817fc-80d7-4d27-b4ac-055953957130 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:41:44.314404 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e817fc-80d7-4d27-b4ac-055953957130 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:41:44.340641 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-e4e817fc-80d7-4d27-b4ac-055953957130 None None] MFA Rules not processed for user `ea0fee783782439ebc8bcd0b957b6459`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:41:44.341844 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-e4e817fc-80d7-4d27-b4ac-055953957130 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:41:44.375483 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:41:44.406060 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1431/2855] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:41:44 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:41:44.411235 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a35a9b1-c5c2-4f43-8a03-7ae0a4018a99 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:41:44.431698 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:DELETE=8,keystone:SELECT=89,keystone:INSERT=3 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:41:44.473138 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a35a9b1-c5c2-4f43-8a03-7ae0a4018a99 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:41:44.477742 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faab879e-8472-48c3-a2b1-09a6228ec361 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:41:44.478425 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faab879e-8472-48c3-a2b1-09a6228ec361 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:41:44.478712 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faab879e-8472-48c3-a2b1-09a6228ec361 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:41:44.478990 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faab879e-8472-48c3-a2b1-09a6228ec361 admin admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:41:44.479984 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faab879e-8472-48c3-a2b1-09a6228ec361 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:41:44.481912 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faab879e-8472-48c3-a2b1-09a6228ec361 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:41:44.590490 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1425/2856] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jun 29 11:41:44 2022] POST /identity/v3/projects => generated 387 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 11:41:44.594438 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e817fc-80d7-4d27-b4ac-055953957130 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:41:44.605909 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e817fc-80d7-4d27-b4ac-055953957130 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:41:44.670430 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5173335a-f3d5-4d69-a893-7690f406aa97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:41:44.671112 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5173335a-f3d5-4d69-a893-7690f406aa97 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:41:44.671398 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5173335a-f3d5-4d69-a893-7690f406aa97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:41:44.671678 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5173335a-f3d5-4d69-a893-7690f406aa97 admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:41:44.672407 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5173335a-f3d5-4d69-a893-7690f406aa97 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:41:44.674312 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5173335a-f3d5-4d69-a893-7690f406aa97 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:41:44.800656 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1432/2857] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jun 29 11:41:44 2022] POST /identity/v3/users => generated 300 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 11:41:44.804672 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faab879e-8472-48c3-a2b1-09a6228ec361 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:41:44.815478 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faab879e-8472-48c3-a2b1-09a6228ec361 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:41:44.819965 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad6ada2-7aef-4e11-b573-c002c099c4a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:41:44.820614 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad6ada2-7aef-4e11-b573-c002c099c4a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:41:44.820894 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad6ada2-7aef-4e11-b573-c002c099c4a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:41:44.821168 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad6ada2-7aef-4e11-b573-c002c099c4a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:41:44.821921 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad6ada2-7aef-4e11-b573-c002c099c4a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:41:44.823699 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad6ada2-7aef-4e11-b573-c002c099c4a4 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:41:44.835813 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1426/2858] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:41:44 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 11:41:44.839787 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5173335a-f3d5-4d69-a893-7690f406aa97 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:41:44.852157 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5173335a-f3d5-4d69-a893-7690f406aa97 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:41:44.854242 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7720e49b-6bf5-4933-b514-ed7f4f700990 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:41:44.855149 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7720e49b-6bf5-4933-b514-ed7f4f700990 admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:41:44.855431 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7720e49b-6bf5-4933-b514-ed7f4f700990 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:41:44.855705 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7720e49b-6bf5-4933-b514-ed7f4f700990 admin admin] PATH_INFO: `/v3/projects/927900f4b65c4cdb82cf480645839a32/users/136044c382b240e489b6f6ef5b099ed2/roles/e95c2ce63e114cc19a9dda6ad21e794f` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:41:44.870599 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7720e49b-6bf5-4933-b514-ed7f4f700990 admin admin] RBAC: Authorizing `identity:create_grant(project_id=927900f4b65c4cdb82cf480645839a32, user_id=136044c382b240e489b6f6ef5b099ed2, role_id=e95c2ce63e114cc19a9dda6ad21e794f)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:41:44.873353 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7720e49b-6bf5-4933-b514-ed7f4f700990 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:41:44.966602 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1433/2859] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jun 29 11:41:44 2022] PUT /identity/v3/projects/927900f4b65c4cdb82cf480645839a32/users/136044c382b240e489b6f6ef5b099ed2/roles/e95c2ce63e114cc19a9dda6ad21e794f => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:41:44.970519 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad6ada2-7aef-4e11-b573-c002c099c4a4 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:41:44.980422 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad6ada2-7aef-4e11-b573-c002c099c4a4 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:41:45.034905 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50223828-64db-421e-95b4-d5a8bc094c88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:41:45.035658 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50223828-64db-421e-95b4-d5a8bc094c88 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:41:45.035974 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50223828-64db-421e-95b4-d5a8bc094c88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:41:45.036254 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50223828-64db-421e-95b4-d5a8bc094c88 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:41:45.036962 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50223828-64db-421e-95b4-d5a8bc094c88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:41:45.038578 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50223828-64db-421e-95b4-d5a8bc094c88 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:41:45.048114 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1427/2860] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:41:44 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 11:41:45.052595 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7720e49b-6bf5-4933-b514-ed7f4f700990 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:41:45.062614 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7720e49b-6bf5-4933-b514-ed7f4f700990 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:41:45.067765 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e725e69-7409-4b0b-8b7e-7cdb4a4cfbe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:41:45.068661 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e725e69-7409-4b0b-8b7e-7cdb4a4cfbe9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:41:45.068966 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e725e69-7409-4b0b-8b7e-7cdb4a4cfbe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:41:45.069238 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e725e69-7409-4b0b-8b7e-7cdb4a4cfbe9 admin admin] PATH_INFO: `/v3/projects/927900f4b65c4cdb82cf480645839a32/users/136044c382b240e489b6f6ef5b099ed2/roles/e95c2ce63e114cc19a9dda6ad21e794f` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:41:45.071172 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e725e69-7409-4b0b-8b7e-7cdb4a4cfbe9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=927900f4b65c4cdb82cf480645839a32, user_id=136044c382b240e489b6f6ef5b099ed2, role_id=e95c2ce63e114cc19a9dda6ad21e794f)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:41:45.073801 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e725e69-7409-4b0b-8b7e-7cdb4a4cfbe9 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:41:45.084088 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1434/2861] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jun 29 11:41:45 2022] PUT /identity/v3/projects/927900f4b65c4cdb82cf480645839a32/users/136044c382b240e489b6f6ef5b099ed2/roles/e95c2ce63e114cc19a9dda6ad21e794f => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:41:45.093979 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccff260-447e-4574-ab05-4fed9f1c52b6 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:41:45.094272 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccff260-447e-4574-ab05-4fed9f1c52b6 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:41:45.094555 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccff260-447e-4574-ab05-4fed9f1c52b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:41:45.120680 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-cccff260-447e-4574-ab05-4fed9f1c52b6 None None] MFA Rules not processed for user `136044c382b240e489b6f6ef5b099ed2`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:41:45.121887 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-cccff260-447e-4574-ab05-4fed9f1c52b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:41:45.187546 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1428/2862] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:41:45 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:41:45.199160 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e725e69-7409-4b0b-8b7e-7cdb4a4cfbe9 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:41:45.200889 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-0e725e69-7409-4b0b-8b7e-7cdb4a4cfbe9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:41:45.257068 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e725e69-7409-4b0b-8b7e-7cdb4a4cfbe9 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:41:45.260715 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6511df2-d0a5-46a5-a386-9cec87de53f4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff7931b75a924394b1af733febce0834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:41:45.261349 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6511df2-d0a5-46a5-a386-9cec87de53f4 service cinder] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:41:45.261662 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6511df2-d0a5-46a5-a386-9cec87de53f4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:41:45.261967 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6511df2-d0a5-46a5-a386-9cec87de53f4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:41:45.270758 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6511df2-d0a5-46a5-a386-9cec87de53f4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:41:45.272192 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6511df2-d0a5-46a5-a386-9cec87de53f4 service cinder] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:41:45.281452 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1435/2863] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:41:45 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:41:45.309436 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccff260-447e-4574-ab05-4fed9f1c52b6 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:41:45.326789 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:41:45.369981 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccff260-447e-4574-ab05-4fed9f1c52b6 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:41:45.374470 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26f12e7-fc63-417d-8750-ab9b2279f26a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:41:45.375326 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26f12e7-fc63-417d-8750-ab9b2279f26a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:41:45.375602 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26f12e7-fc63-417d-8750-ab9b2279f26a admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:41:45.375895 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26f12e7-fc63-417d-8750-ab9b2279f26a admin admin] PATH_INFO: `/v3/users/ea0fee783782439ebc8bcd0b957b6459` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:41:45.377227 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26f12e7-fc63-417d-8750-ab9b2279f26a admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea0fee783782439ebc8bcd0b957b6459)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:41:45.379709 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26f12e7-fc63-417d-8750-ab9b2279f26a admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:41:45.597950 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-b26f12e7-fc63-417d-8750-ab9b2279f26a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea0fee783782439ebc8bcd0b957b6459'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:41:45.680620 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-b26f12e7-fc63-417d-8750-ab9b2279f26a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea0fee783782439ebc8bcd0b957b6459'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:41:45.681298 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-b26f12e7-fc63-417d-8750-ab9b2279f26a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea0fee783782439ebc8bcd0b957b6459'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:41:45.693613 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-b26f12e7-fc63-417d-8750-ab9b2279f26a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea0fee783782439ebc8bcd0b957b6459'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:41:45.694280 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-b26f12e7-fc63-417d-8750-ab9b2279f26a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea0fee783782439ebc8bcd0b957b6459'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:41:45.706628 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1429/2864] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jun 29 11:41:45 2022] DELETE /identity/v3/users/ea0fee783782439ebc8bcd0b957b6459 => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:41:45.711153 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6685c1-84d9-4905-ad2a-0d7b955ee015 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:41:45.711642 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6685c1-84d9-4905-ad2a-0d7b955ee015 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:41:45.711878 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6685c1-84d9-4905-ad2a-0d7b955ee015 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:41:45.729997 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-eb6685c1-84d9-4905-ad2a-0d7b955ee015 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:41:45.731306 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-eb6685c1-84d9-4905-ad2a-0d7b955ee015 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:41:45.758944 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:41:45.795019 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1436/2865] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:41:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:41:45.805936 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26f12e7-fc63-417d-8750-ab9b2279f26a admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:41:45.807558 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-b26f12e7-fc63-417d-8750-ab9b2279f26a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:41:45.868804 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26f12e7-fc63-417d-8750-ab9b2279f26a admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:41:45.872073 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cbef5b-3b61-4c0d-a1d3-50020b0e33fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f4bb9f989e4024a5da28dced646b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:41:45.872852 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cbef5b-3b61-4c0d-a1d3-50020b0e33fb service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:41:45.873138 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cbef5b-3b61-4c0d-a1d3-50020b0e33fb service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:41:45.873410 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cbef5b-3b61-4c0d-a1d3-50020b0e33fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:41:45.882840 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cbef5b-3b61-4c0d-a1d3-50020b0e33fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:41:45.884356 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cbef5b-3b61-4c0d-a1d3-50020b0e33fb service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:41:45.894977 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1430/2866] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:41:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:41:46.339804 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6685c1-84d9-4905-ad2a-0d7b955ee015 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:41:46.341486 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-eb6685c1-84d9-4905-ad2a-0d7b955ee015 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:41:46.354445 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6685c1-84d9-4905-ad2a-0d7b955ee015 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:41:46.359197 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13d43fd-74f3-4068-8d1d-4e7f57d2b4f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:41:46.360057 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13d43fd-74f3-4068-8d1d-4e7f57d2b4f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:41:46.360339 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13d43fd-74f3-4068-8d1d-4e7f57d2b4f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:41:46.360644 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13d43fd-74f3-4068-8d1d-4e7f57d2b4f6 admin admin] PATH_INFO: `/v3/projects/8107c80665bd4171a81f22edb640a432` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:41:46.361925 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13d43fd-74f3-4068-8d1d-4e7f57d2b4f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8107c80665bd4171a81f22edb640a432)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:41:46.364691 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13d43fd-74f3-4068-8d1d-4e7f57d2b4f6 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:41:46.474895 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-c13d43fd-74f3-4068-8d1d-4e7f57d2b4f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8107c80665bd4171a81f22edb640a432', 'request_id': 'req-c13d43fd-74f3-4068-8d1d-4e7f57d2b4f6'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:41:46.481151 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-c13d43fd-74f3-4068-8d1d-4e7f57d2b4f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8107c80665bd4171a81f22edb640a432', 'request_id': 'req-c13d43fd-74f3-4068-8d1d-4e7f57d2b4f6'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:41:46.486724 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-c13d43fd-74f3-4068-8d1d-4e7f57d2b4f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8107c80665bd4171a81f22edb640a432', 'request_id': 'req-c13d43fd-74f3-4068-8d1d-4e7f57d2b4f6'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:41:46.640370 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-c13d43fd-74f3-4068-8d1d-4e7f57d2b4f6 admin admin] The token cache is being invalidate because project 8107c80665bd4171a81f22edb640a432 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 11:41:46.640668 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-c13d43fd-74f3-4068-8d1d-4e7f57d2b4f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:41:46.641292 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-c13d43fd-74f3-4068-8d1d-4e7f57d2b4f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:41:46.644689 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1437/2867] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jun 29 11:41:46 2022] DELETE /identity/v3/projects/8107c80665bd4171a81f22edb640a432 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:41:46.647391 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cbef5b-3b61-4c0d-a1d3-50020b0e33fb service neutron] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:41:46.657225 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cbef5b-3b61-4c0d-a1d3-50020b0e33fb service neutron] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:41:46.693804 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:41:46.717532 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467ab779-67d8-44d1-8788-3a44a165b481 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:41:46.719422 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467ab779-67d8-44d1-8788-3a44a165b481 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:41:46.719422 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467ab779-67d8-44d1-8788-3a44a165b481 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:41:46.719422 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467ab779-67d8-44d1-8788-3a44a165b481 admin admin] PATH_INFO: `/v3/users/136044c382b240e489b6f6ef5b099ed2` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:41:46.720459 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467ab779-67d8-44d1-8788-3a44a165b481 admin admin] RBAC: Authorizing `identity:delete_user(user_id=136044c382b240e489b6f6ef5b099ed2)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:41:46.723128 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467ab779-67d8-44d1-8788-3a44a165b481 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:41:46.754867 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13d43fd-74f3-4068-8d1d-4e7f57d2b4f6 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:41:46.756334 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-c13d43fd-74f3-4068-8d1d-4e7f57d2b4f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:41:46.828886 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13d43fd-74f3-4068-8d1d-4e7f57d2b4f6 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:41:46.834434 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a8d335-398c-497a-a18d-c953c2ad2ec3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a0505342b704d02977a5125957c2c9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:41:46.835086 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a8d335-398c-497a-a18d-c953c2ad2ec3 service placement] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:41:46.835371 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a8d335-398c-497a-a18d-c953c2ad2ec3 service placement] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:41:46.839736 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a8d335-398c-497a-a18d-c953c2ad2ec3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:41:46.839736 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-28a8d335-398c-497a-a18d-c953c2ad2ec3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:41:46.849447 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a8d335-398c-497a-a18d-c953c2ad2ec3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:41:46.851032 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a8d335-398c-497a-a18d-c953c2ad2ec3 service placement] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:41:46.860935 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1438/2868] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:41:46 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 11:41:46.985498 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-467ab779-67d8-44d1-8788-3a44a165b481 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '136044c382b240e489b6f6ef5b099ed2'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:41:47.125118 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-467ab779-67d8-44d1-8788-3a44a165b481 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '136044c382b240e489b6f6ef5b099ed2'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:41:47.125768 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-467ab779-67d8-44d1-8788-3a44a165b481 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '136044c382b240e489b6f6ef5b099ed2'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:41:47.140400 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-467ab779-67d8-44d1-8788-3a44a165b481 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '136044c382b240e489b6f6ef5b099ed2'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:41:47.141044 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-467ab779-67d8-44d1-8788-3a44a165b481 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '136044c382b240e489b6f6ef5b099ed2'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:41:47.153599 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1431/2869] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jun 29 11:41:46 2022] DELETE /identity/v3/users/136044c382b240e489b6f6ef5b099ed2 => generated 0 bytes in 507 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:41:47.819652 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a8d335-398c-497a-a18d-c953c2ad2ec3 service placement] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:41:47.821267 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-28a8d335-398c-497a-a18d-c953c2ad2ec3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:41:47.835210 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a8d335-398c-497a-a18d-c953c2ad2ec3 service placement] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:41:47.882307 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:41:47.897857 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd8669b-92cd-4369-884e-fc545da3a4cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:41:47.898720 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd8669b-92cd-4369-884e-fc545da3a4cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:41:47.898998 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd8669b-92cd-4369-884e-fc545da3a4cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:41:47.899271 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd8669b-92cd-4369-884e-fc545da3a4cc admin admin] PATH_INFO: `/v3/projects/927900f4b65c4cdb82cf480645839a32` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:41:47.900466 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd8669b-92cd-4369-884e-fc545da3a4cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=927900f4b65c4cdb82cf480645839a32)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:41:47.902349 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd8669b-92cd-4369-884e-fc545da3a4cc admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:41:48.052104 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-6cd8669b-92cd-4369-884e-fc545da3a4cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '927900f4b65c4cdb82cf480645839a32', 'request_id': 'req-6cd8669b-92cd-4369-884e-fc545da3a4cc'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:41:48.058180 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-6cd8669b-92cd-4369-884e-fc545da3a4cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '927900f4b65c4cdb82cf480645839a32', 'request_id': 'req-6cd8669b-92cd-4369-884e-fc545da3a4cc'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:41:48.063505 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-6cd8669b-92cd-4369-884e-fc545da3a4cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '927900f4b65c4cdb82cf480645839a32', 'request_id': 'req-6cd8669b-92cd-4369-884e-fc545da3a4cc'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:41:48.177719 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-6cd8669b-92cd-4369-884e-fc545da3a4cc admin admin] The token cache is being invalidate because project 927900f4b65c4cdb82cf480645839a32 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 11:41:48.178009 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-6cd8669b-92cd-4369-884e-fc545da3a4cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:41:48.178623 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-6cd8669b-92cd-4369-884e-fc545da3a4cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:41:48.182872 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1439/2870] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jun 29 11:41:47 2022] DELETE /identity/v3/projects/927900f4b65c4cdb82cf480645839a32 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:41:48.185080 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5aaf11-6d48-411f-804b-5d8209ab181d None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:41:48.185545 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5aaf11-6d48-411f-804b-5d8209ab181d None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:41:48.185651 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5aaf11-6d48-411f-804b-5d8209ab181d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:41:48.203758 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-9e5aaf11-6d48-411f-804b-5d8209ab181d None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:41:48.206154 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-9e5aaf11-6d48-411f-804b-5d8209ab181d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:41:48.270437 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1432/2871] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:41:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:41:48.279168 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75dee12-ad77-4201-b231-c7fc686d64f2 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:41:48.279454 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75dee12-ad77-4201-b231-c7fc686d64f2 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:41:48.279754 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75dee12-ad77-4201-b231-c7fc686d64f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:41:48.298329 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-b75dee12-ad77-4201-b231-c7fc686d64f2 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:41:48.300119 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-b75dee12-ad77-4201-b231-c7fc686d64f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:41:48.308135 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1440/2872] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:41:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:41:48.317919 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5aaf11-6d48-411f-804b-5d8209ab181d None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:41:48.327560 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5aaf11-6d48-411f-804b-5d8209ab181d None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:41:48.332077 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e819da5-1190-42ff-866c-55a5243576c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:41:48.332757 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e819da5-1190-42ff-866c-55a5243576c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:41:48.333040 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e819da5-1190-42ff-866c-55a5243576c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:41:48.333316 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e819da5-1190-42ff-866c-55a5243576c3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:41:48.334040 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e819da5-1190-42ff-866c-55a5243576c3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80165) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:41:48.334353 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e819da5-1190-42ff-866c-55a5243576c3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:41:48.335964 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e819da5-1190-42ff-866c-55a5243576c3 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:41:48.347297 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1433/2873] 10.222.0.44 () {58 vars in 1152 bytes} [Wed Jun 29 11:41:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:41:48.351518 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75dee12-ad77-4201-b231-c7fc686d64f2 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:41:48.361467 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75dee12-ad77-4201-b231-c7fc686d64f2 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:41:48.367529 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59165e0-e6f2-46ac-87fb-08e0d077dca7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:41:48.367804 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59165e0-e6f2-46ac-87fb-08e0d077dca7 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:41:48.368136 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59165e0-e6f2-46ac-87fb-08e0d077dca7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:41:48.368413 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59165e0-e6f2-46ac-87fb-08e0d077dca7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:41:48.369108 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59165e0-e6f2-46ac-87fb-08e0d077dca7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:41:48.371092 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59165e0-e6f2-46ac-87fb-08e0d077dca7 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:41:48.539641 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1441/2874] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jun 29 11:41:48 2022] POST /identity/v3/projects => generated 373 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:41:48.543643 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e819da5-1190-42ff-866c-55a5243576c3 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:41:48.553542 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e819da5-1190-42ff-866c-55a5243576c3 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:41:48.596322 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:41:48.611190 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9422e4c7-a839-4390-a3f4-ddbefd286b0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:41:48.611886 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9422e4c7-a839-4390-a3f4-ddbefd286b0a admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:41:48.612178 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9422e4c7-a839-4390-a3f4-ddbefd286b0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:41:48.612456 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9422e4c7-a839-4390-a3f4-ddbefd286b0a admin admin] PATH_INFO: `/v3/users` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:41:48.613402 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9422e4c7-a839-4390-a3f4-ddbefd286b0a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:41:48.615320 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9422e4c7-a839-4390-a3f4-ddbefd286b0a admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:41:48.810158 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1434/2875] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jun 29 11:41:48 2022] POST /identity/v3/users => generated 287 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:41:48.811387 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59165e0-e6f2-46ac-87fb-08e0d077dca7 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:41:48.822310 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59165e0-e6f2-46ac-87fb-08e0d077dca7 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:41:48.828799 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f4fdd2-371a-4b8d-9362-356f5706eac0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:41:48.828799 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f4fdd2-371a-4b8d-9362-356f5706eac0 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:41:48.828799 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f4fdd2-371a-4b8d-9362-356f5706eac0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:41:48.828799 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f4fdd2-371a-4b8d-9362-356f5706eac0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:41:48.829206 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f4fdd2-371a-4b8d-9362-356f5706eac0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:41:48.830532 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f4fdd2-371a-4b8d-9362-356f5706eac0 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:41:48.841074 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1442/2876] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:41:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 11:41:48.845156 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9422e4c7-a839-4390-a3f4-ddbefd286b0a admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:41:48.855716 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9422e4c7-a839-4390-a3f4-ddbefd286b0a admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:41:48.860221 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7300441a-fb53-4d00-9ea5-f6a5efc74a96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:41:48.860870 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7300441a-fb53-4d00-9ea5-f6a5efc74a96 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:41:48.861150 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7300441a-fb53-4d00-9ea5-f6a5efc74a96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:41:48.861426 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7300441a-fb53-4d00-9ea5-f6a5efc74a96 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:41:48.862986 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7300441a-fb53-4d00-9ea5-f6a5efc74a96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:41:48.864605 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7300441a-fb53-4d00-9ea5-f6a5efc74a96 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:41:48.876456 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1435/2877] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:41:48 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 11:41:48.881581 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f4fdd2-371a-4b8d-9362-356f5706eac0 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:41:48.892199 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f4fdd2-371a-4b8d-9362-356f5706eac0 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:41:48.897591 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135927c4-74dd-4abb-8d27-5d72670ed3a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:41:48.898362 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135927c4-74dd-4abb-8d27-5d72670ed3a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:41:48.898751 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135927c4-74dd-4abb-8d27-5d72670ed3a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:41:48.899032 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135927c4-74dd-4abb-8d27-5d72670ed3a7 admin admin] PATH_INFO: `/v3/projects/b0f099825879470bba2817a0d02c0e84/users/6271b17d93a947beb71b5a42e70db55a/roles/e1c88eebe911432cab6a07fbeb84653b` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:41:48.921783 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135927c4-74dd-4abb-8d27-5d72670ed3a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0f099825879470bba2817a0d02c0e84, user_id=6271b17d93a947beb71b5a42e70db55a, role_id=e1c88eebe911432cab6a07fbeb84653b)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:41:48.924997 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135927c4-74dd-4abb-8d27-5d72670ed3a7 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:41:49.011455 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1443/2878] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jun 29 11:41:48 2022] PUT /identity/v3/projects/b0f099825879470bba2817a0d02c0e84/users/6271b17d93a947beb71b5a42e70db55a/roles/e1c88eebe911432cab6a07fbeb84653b => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:41:49.016696 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93aa9db8-a922-49f4-bb92-ac00d8e64379 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:41:49.016984 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93aa9db8-a922-49f4-bb92-ac00d8e64379 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:41:49.017256 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93aa9db8-a922-49f4-bb92-ac00d8e64379 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:41:49.034205 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-93aa9db8-a922-49f4-bb92-ac00d8e64379 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:41:49.035402 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-93aa9db8-a922-49f4-bb92-ac00d8e64379 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:41:49.092380 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1436/2879] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:41:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 11:41:49.102025 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135927c4-74dd-4abb-8d27-5d72670ed3a7 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:41:49.103746 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-135927c4-74dd-4abb-8d27-5d72670ed3a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:41:49.159951 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135927c4-74dd-4abb-8d27-5d72670ed3a7 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:41:49.163517 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac40b490-93e1-4140-8c2c-de525761792e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f4bb9f989e4024a5da28dced646b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:41:49.164197 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac40b490-93e1-4140-8c2c-de525761792e service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:41:49.164489 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac40b490-93e1-4140-8c2c-de525761792e service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:41:49.164763 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac40b490-93e1-4140-8c2c-de525761792e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:41:49.173611 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac40b490-93e1-4140-8c2c-de525761792e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:41:49.175708 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac40b490-93e1-4140-8c2c-de525761792e service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:41:49.185014 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1444/2880] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:41:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:41:51.007659 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695389ae-d0f2-47bb-9f1b-93fc3098cff6 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:41:51.014207 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695389ae-d0f2-47bb-9f1b-93fc3098cff6 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:41:51.014207 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695389ae-d0f2-47bb-9f1b-93fc3098cff6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:41:51.037005 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-695389ae-d0f2-47bb-9f1b-93fc3098cff6 None None] MFA Rules not processed for user `6271b17d93a947beb71b5a42e70db55a`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:41:51.046679 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-695389ae-d0f2-47bb-9f1b-93fc3098cff6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:41:51.141381 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1437/2881] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:41:51 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:41:51.148670 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac40b490-93e1-4140-8c2c-de525761792e service neutron] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:41:51.150168 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-ac40b490-93e1-4140-8c2c-de525761792e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:41:51.167017 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:41:51.214965 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac40b490-93e1-4140-8c2c-de525761792e service neutron] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:41:51.219095 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15a16f3-dc04-4335-b509-a831c80308f6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff7931b75a924394b1af733febce0834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:41:51.219864 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15a16f3-dc04-4335-b509-a831c80308f6 service cinder] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:41:51.220153 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15a16f3-dc04-4335-b509-a831c80308f6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:41:51.220427 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15a16f3-dc04-4335-b509-a831c80308f6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:41:51.229711 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15a16f3-dc04-4335-b509-a831c80308f6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:41:51.231136 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15a16f3-dc04-4335-b509-a831c80308f6 service cinder] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:41:51.240803 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1445/2882] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:41:51 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:41:51.744491 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695389ae-d0f2-47bb-9f1b-93fc3098cff6 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:41:51.750446 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-695389ae-d0f2-47bb-9f1b-93fc3098cff6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:41:51.816865 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:41:51.838620 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695389ae-d0f2-47bb-9f1b-93fc3098cff6 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:41:51.844223 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c77b54-8712-4fee-9dcc-acdab7652e30 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '796aeeff16df4739bddec1f6dee61bfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:41:51.845593 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c77b54-8712-4fee-9dcc-acdab7652e30 service nova] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:41:51.845887 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c77b54-8712-4fee-9dcc-acdab7652e30 service nova] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:41:51.846256 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c77b54-8712-4fee-9dcc-acdab7652e30 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:41:51.848269 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-91c77b54-8712-4fee-9dcc-acdab7652e30 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:41:51.866382 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c77b54-8712-4fee-9dcc-acdab7652e30 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:41:51.868191 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c77b54-8712-4fee-9dcc-acdab7652e30 service nova] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:41:51.883166 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1438/2883] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:41:51 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:41:54.415858 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f515f2eb-4e1a-4aa5-9f1f-34d92920baae None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:41:54.416355 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f515f2eb-4e1a-4aa5-9f1f-34d92920baae None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:41:54.416468 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f515f2eb-4e1a-4aa5-9f1f-34d92920baae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:41:54.433877 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-f515f2eb-4e1a-4aa5-9f1f-34d92920baae None None] MFA Rules not processed for user `6271b17d93a947beb71b5a42e70db55a`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:41:54.436172 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-f515f2eb-4e1a-4aa5-9f1f-34d92920baae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:41:54.441974 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1446/2884] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:41:54 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:41:54.669388 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c77b54-8712-4fee-9dcc-acdab7652e30 service nova] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:41:54.680026 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c77b54-8712-4fee-9dcc-acdab7652e30 service nova] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:41:54.683586 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f12ec3-4758-468f-b54f-7d86e0451f8a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f4bb9f989e4024a5da28dced646b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:41:54.686082 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f12ec3-4758-468f-b54f-7d86e0451f8a service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:41:54.686374 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f12ec3-4758-468f-b54f-7d86e0451f8a service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:41:54.686664 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f12ec3-4758-468f-b54f-7d86e0451f8a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:41:54.692304 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-69f12ec3-4758-468f-b54f-7d86e0451f8a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:41:54.712728 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f12ec3-4758-468f-b54f-7d86e0451f8a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:41:54.712728 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f12ec3-4758-468f-b54f-7d86e0451f8a service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:41:54.719811 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1439/2885] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:41:54 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 11:41:59.598377 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f515f2eb-4e1a-4aa5-9f1f-34d92920baae None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:41:59.600164 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-f515f2eb-4e1a-4aa5-9f1f-34d92920baae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:41:59.618752 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f515f2eb-4e1a-4aa5-9f1f-34d92920baae None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:41:59.624583 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9190899-23d7-4fe3-9b4e-22b01f33dbfe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '796aeeff16df4739bddec1f6dee61bfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:41:59.625243 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9190899-23d7-4fe3-9b4e-22b01f33dbfe service nova] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:41:59.625534 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9190899-23d7-4fe3-9b4e-22b01f33dbfe service nova] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:41:59.625819 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9190899-23d7-4fe3-9b4e-22b01f33dbfe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:41:59.627685 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-f9190899-23d7-4fe3-9b4e-22b01f33dbfe service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:41:59.641348 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9190899-23d7-4fe3-9b4e-22b01f33dbfe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:41:59.642957 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9190899-23d7-4fe3-9b4e-22b01f33dbfe service nova] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:41:59.655171 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1447/2886] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:41:59 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:42:02.764645 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f12ec3-4758-468f-b54f-7d86e0451f8a service neutron] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:42:02.773683 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f12ec3-4758-468f-b54f-7d86e0451f8a service neutron] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:42:02.778135 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ad20d5-d687-41f1-9984-5d76989dae06 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f4bb9f989e4024a5da28dced646b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:42:02.778788 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ad20d5-d687-41f1-9984-5d76989dae06 service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:42:02.779046 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ad20d5-d687-41f1-9984-5d76989dae06 service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:42:02.779320 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ad20d5-d687-41f1-9984-5d76989dae06 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:42:02.781202 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-42ad20d5-d687-41f1-9984-5d76989dae06 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:42:02.844511 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ad20d5-d687-41f1-9984-5d76989dae06 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:42:02.846066 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ad20d5-d687-41f1-9984-5d76989dae06 service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:42:02.856349 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1440/2887] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:42:02 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:42:09.650093 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=29 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:42:12.852057 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=46 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:42:14.494533 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0d4bad-43a1-402f-bcd9-09463c9605e7 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:42:14.495090 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0d4bad-43a1-402f-bcd9-09463c9605e7 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:42:14.495090 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0d4bad-43a1-402f-bcd9-09463c9605e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:42:14.512329 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-5b0d4bad-43a1-402f-bcd9-09463c9605e7 None None] MFA Rules not processed for user `6271b17d93a947beb71b5a42e70db55a`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:42:14.513525 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-5b0d4bad-43a1-402f-bcd9-09463c9605e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:42:14.520861 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1448/2888] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:42:14 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 11:42:24.507480 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=3 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:42:40.624795 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f53c683-9d2f-4e7a-9efe-d8b207ff42de None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:42:40.625292 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f53c683-9d2f-4e7a-9efe-d8b207ff42de None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:42:40.625410 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f53c683-9d2f-4e7a-9efe-d8b207ff42de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:42:40.684271 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-8f53c683-9d2f-4e7a-9efe-d8b207ff42de None None] MFA Rules not processed for user `55487c2107a44c15b656b4865ee0d4fa`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:42:40.685519 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-8f53c683-9d2f-4e7a-9efe-d8b207ff42de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:42:40.751093 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1441/2889] 10.222.0.44 () {58 vars in 1009 bytes} [Wed Jun 29 11:42:40 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:42:40.935868 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0d4bad-43a1-402f-bcd9-09463c9605e7 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:42:40.941557 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-5b0d4bad-43a1-402f-bcd9-09463c9605e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:42:41.035700 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0d4bad-43a1-402f-bcd9-09463c9605e7 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:42:41.040920 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20fae6a-b2d2-41ed-8a0d-498d9d16abf0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3b38493ba904759badc800e396befda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:42:41.041581 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20fae6a-b2d2-41ed-8a0d-498d9d16abf0 service swift] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:42:41.041866 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20fae6a-b2d2-41ed-8a0d-498d9d16abf0 service swift] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:42:41.042146 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20fae6a-b2d2-41ed-8a0d-498d9d16abf0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:42:41.050955 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20fae6a-b2d2-41ed-8a0d-498d9d16abf0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:42:41.052613 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20fae6a-b2d2-41ed-8a0d-498d9d16abf0 service swift] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:42:41.060483 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1449/2890] 10.222.0.44 () {60 vars in 1384 bytes} [Wed Jun 29 11:42:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 29 11:42:50.747966 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=44 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:42:51.057438 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=41 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:42:57.858386 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0085e8f-c26e-45c1-8893-ce60a87def7b None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:42:57.858886 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0085e8f-c26e-45c1-8893-ce60a87def7b None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:42:57.858995 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0085e8f-c26e-45c1-8893-ce60a87def7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:42:57.876706 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-f0085e8f-c26e-45c1-8893-ce60a87def7b None None] MFA Rules not processed for user `55487c2107a44c15b656b4865ee0d4fa`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:42:57.878389 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-f0085e8f-c26e-45c1-8893-ce60a87def7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:42:57.885359 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1442/2891] 10.222.0.44 () {58 vars in 967 bytes} [Wed Jun 29 11:42:57 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:42:57.897803 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20fae6a-b2d2-41ed-8a0d-498d9d16abf0 service swift] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:42:57.908231 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20fae6a-b2d2-41ed-8a0d-498d9d16abf0 service swift] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:42:57.913266 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f47c9f4-2317-434a-91f0-5b310294c6ab service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3b38493ba904759badc800e396befda', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:42:57.913908 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f47c9f4-2317-434a-91f0-5b310294c6ab service swift] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:42:57.914196 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f47c9f4-2317-434a-91f0-5b310294c6ab service swift] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:42:57.914477 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f47c9f4-2317-434a-91f0-5b310294c6ab service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:42:57.922911 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f47c9f4-2317-434a-91f0-5b310294c6ab service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:42:57.924553 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f47c9f4-2317-434a-91f0-5b310294c6ab service swift] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:42:57.932723 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1450/2892] 10.222.0.44 () {60 vars in 1384 bytes} [Wed Jun 29 11:42:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 29 11:43:01.101388 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0085e8f-c26e-45c1-8893-ce60a87def7b None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:43:01.102896 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-f0085e8f-c26e-45c1-8893-ce60a87def7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:43:01.173803 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0085e8f-c26e-45c1-8893-ce60a87def7b None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:43:01.178813 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2e1c58-570c-430f-a44d-980524b81964 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a0505342b704d02977a5125957c2c9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:43:01.179451 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2e1c58-570c-430f-a44d-980524b81964 service placement] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:43:01.179730 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2e1c58-570c-430f-a44d-980524b81964 service placement] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:43:01.180042 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2e1c58-570c-430f-a44d-980524b81964 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:43:01.181829 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-3d2e1c58-570c-430f-a44d-980524b81964 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:43:01.193145 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2e1c58-570c-430f-a44d-980524b81964 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:43:01.194740 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2e1c58-570c-430f-a44d-980524b81964 service placement] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:43:01.204447 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1443/2893] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:43:01 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:43:01.236056 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f47c9f4-2317-434a-91f0-5b310294c6ab service swift] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:43:01.245084 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f47c9f4-2317-434a-91f0-5b310294c6ab service swift] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:43:01.250064 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fc0f1b-dca0-41c1-9ae6-2565317f27c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:43:01.250940 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fc0f1b-dca0-41c1-9ae6-2565317f27c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:43:01.251223 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fc0f1b-dca0-41c1-9ae6-2565317f27c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:43:01.251499 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fc0f1b-dca0-41c1-9ae6-2565317f27c7 admin admin] PATH_INFO: `/v3/users/6271b17d93a947beb71b5a42e70db55a` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:43:01.252745 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fc0f1b-dca0-41c1-9ae6-2565317f27c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6271b17d93a947beb71b5a42e70db55a)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:43:01.254704 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fc0f1b-dca0-41c1-9ae6-2565317f27c7 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:43:01.485901 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-04fc0f1b-dca0-41c1-9ae6-2565317f27c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6271b17d93a947beb71b5a42e70db55a'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:43:01.486554 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-04fc0f1b-dca0-41c1-9ae6-2565317f27c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6271b17d93a947beb71b5a42e70db55a'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:43:01.581343 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-04fc0f1b-dca0-41c1-9ae6-2565317f27c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6271b17d93a947beb71b5a42e70db55a'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:43:01.588618 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-04fc0f1b-dca0-41c1-9ae6-2565317f27c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6271b17d93a947beb71b5a42e70db55a'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:43:01.589263 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-04fc0f1b-dca0-41c1-9ae6-2565317f27c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6271b17d93a947beb71b5a42e70db55a'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:43:01.606140 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1451/2894] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jun 29 11:43:01 2022] DELETE /identity/v3/users/6271b17d93a947beb71b5a42e70db55a => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:43:01.858473 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2e1c58-570c-430f-a44d-980524b81964 service placement] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:43:01.871942 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2e1c58-570c-430f-a44d-980524b81964 service placement] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:43:01.934809 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dede6c-0dd0-44f3-a000-2dc5b07d8f33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:43:01.935688 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dede6c-0dd0-44f3-a000-2dc5b07d8f33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:43:01.936027 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dede6c-0dd0-44f3-a000-2dc5b07d8f33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:43:01.936292 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dede6c-0dd0-44f3-a000-2dc5b07d8f33 admin admin] PATH_INFO: `/v3/projects/b0f099825879470bba2817a0d02c0e84` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:43:01.937486 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68dede6c-0dd0-44f3-a000-2dc5b07d8f33 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b0f099825879470bba2817a0d02c0e84)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:43:01.940023 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68dede6c-0dd0-44f3-a000-2dc5b07d8f33 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:43:02.043626 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-68dede6c-0dd0-44f3-a000-2dc5b07d8f33 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b0f099825879470bba2817a0d02c0e84', 'request_id': 'req-68dede6c-0dd0-44f3-a000-2dc5b07d8f33'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:43:02.159558 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-68dede6c-0dd0-44f3-a000-2dc5b07d8f33 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b0f099825879470bba2817a0d02c0e84', 'request_id': 'req-68dede6c-0dd0-44f3-a000-2dc5b07d8f33'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:43:02.168845 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-68dede6c-0dd0-44f3-a000-2dc5b07d8f33 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b0f099825879470bba2817a0d02c0e84', 'request_id': 'req-68dede6c-0dd0-44f3-a000-2dc5b07d8f33'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:43:02.180613 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-68dede6c-0dd0-44f3-a000-2dc5b07d8f33 admin admin] The token cache is being invalidate because project b0f099825879470bba2817a0d02c0e84 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 11:43:02.180907 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-68dede6c-0dd0-44f3-a000-2dc5b07d8f33 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:43:02.181584 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-68dede6c-0dd0-44f3-a000-2dc5b07d8f33 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:43:02.183125 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1444/2895] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jun 29 11:43:01 2022] DELETE /identity/v3/projects/b0f099825879470bba2817a0d02c0e84 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:43:02.187995 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad29663-48bb-42ce-b641-17db5baa5cd1 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:43:02.188290 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad29663-48bb-42ce-b641-17db5baa5cd1 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:43:02.188569 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad29663-48bb-42ce-b641-17db5baa5cd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:43:02.207072 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-4ad29663-48bb-42ce-b641-17db5baa5cd1 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:43:02.208326 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-4ad29663-48bb-42ce-b641-17db5baa5cd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:43:02.267091 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1452/2896] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:43:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:43:02.274648 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a69496a-c3fc-4772-8ef7-0ff30e750681 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:43:02.274963 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a69496a-c3fc-4772-8ef7-0ff30e750681 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:43:02.275250 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a69496a-c3fc-4772-8ef7-0ff30e750681 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:43:02.293816 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-0a69496a-c3fc-4772-8ef7-0ff30e750681 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:43:02.295029 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-0a69496a-c3fc-4772-8ef7-0ff30e750681 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:43:02.303585 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1445/2897] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:43:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 11:43:02.307968 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad29663-48bb-42ce-b641-17db5baa5cd1 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:43:02.317593 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad29663-48bb-42ce-b641-17db5baa5cd1 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:43:02.322286 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd848c90-83fb-446c-855d-0eb648d90eda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:43:02.323084 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd848c90-83fb-446c-855d-0eb648d90eda admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:43:02.323378 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd848c90-83fb-446c-855d-0eb648d90eda admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:43:02.323652 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd848c90-83fb-446c-855d-0eb648d90eda admin admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:43:02.324428 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd848c90-83fb-446c-855d-0eb648d90eda admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 11:43:02.324740 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd848c90-83fb-446c-855d-0eb648d90eda admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:43:02.326325 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd848c90-83fb-446c-855d-0eb648d90eda admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:43:02.338455 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1453/2898] 10.222.0.44 () {58 vars in 1152 bytes} [Wed Jun 29 11:43:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 11:43:02.342732 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a69496a-c3fc-4772-8ef7-0ff30e750681 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:43:02.352590 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a69496a-c3fc-4772-8ef7-0ff30e750681 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:43:02.357274 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d0145e-efbe-4909-9e81-575742ef6bf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:43:02.367542 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d0145e-efbe-4909-9e81-575742ef6bf7 admin admin] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:43:02.367542 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d0145e-efbe-4909-9e81-575742ef6bf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:43:02.367542 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d0145e-efbe-4909-9e81-575742ef6bf7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:43:02.367542 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d0145e-efbe-4909-9e81-575742ef6bf7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:43:02.367542 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d0145e-efbe-4909-9e81-575742ef6bf7 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:43:02.367542 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:43:02.509825 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1446/2899] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jun 29 11:43:02 2022] POST /identity/v3/projects => generated 389 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:43:02.514061 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd848c90-83fb-446c-855d-0eb648d90eda admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:43:02.524191 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd848c90-83fb-446c-855d-0eb648d90eda admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:43:02.569016 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:43:02.581761 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2577a6d5-e835-43bc-9d6a-d7034bd72e49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:43:02.582435 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2577a6d5-e835-43bc-9d6a-d7034bd72e49 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:43:02.582721 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2577a6d5-e835-43bc-9d6a-d7034bd72e49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:43:02.582998 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2577a6d5-e835-43bc-9d6a-d7034bd72e49 admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:43:02.583693 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2577a6d5-e835-43bc-9d6a-d7034bd72e49 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:43:02.585664 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2577a6d5-e835-43bc-9d6a-d7034bd72e49 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:43:02.708988 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1454/2900] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jun 29 11:43:02 2022] POST /identity/v3/users => generated 295 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 11:43:02.713141 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d0145e-efbe-4909-9e81-575742ef6bf7 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:43:02.724044 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d0145e-efbe-4909-9e81-575742ef6bf7 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:43:02.728918 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca33150-b3e5-49d2-b94f-af4a39b5e0e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:43:02.729586 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca33150-b3e5-49d2-b94f-af4a39b5e0e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:43:02.729905 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca33150-b3e5-49d2-b94f-af4a39b5e0e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:43:02.730183 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca33150-b3e5-49d2-b94f-af4a39b5e0e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:43:02.730926 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca33150-b3e5-49d2-b94f-af4a39b5e0e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:43:02.732654 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca33150-b3e5-49d2-b94f-af4a39b5e0e6 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:43:02.743173 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1447/2901] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:43:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 11:43:02.747265 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2577a6d5-e835-43bc-9d6a-d7034bd72e49 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:43:02.757435 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2577a6d5-e835-43bc-9d6a-d7034bd72e49 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:43:02.762089 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a85da8-93a8-4404-8d07-274261aeb2de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:43:02.762712 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a85da8-93a8-4404-8d07-274261aeb2de admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:43:02.762992 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a85da8-93a8-4404-8d07-274261aeb2de admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:43:02.763270 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a85da8-93a8-4404-8d07-274261aeb2de admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:43:02.764024 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a85da8-93a8-4404-8d07-274261aeb2de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:43:02.765590 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a85da8-93a8-4404-8d07-274261aeb2de admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:43:02.775892 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1455/2902] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jun 29 11:43:02 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 11:43:02.779950 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca33150-b3e5-49d2-b94f-af4a39b5e0e6 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:43:02.790569 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca33150-b3e5-49d2-b94f-af4a39b5e0e6 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:43:02.795107 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8b4274-9c98-4654-b1f8-bd04e3fe06ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:43:02.797686 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8b4274-9c98-4654-b1f8-bd04e3fe06ac admin admin] REQUEST_METHOD: `PUT` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:43:02.797981 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8b4274-9c98-4654-b1f8-bd04e3fe06ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:43:02.798303 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8b4274-9c98-4654-b1f8-bd04e3fe06ac admin admin] PATH_INFO: `/v3/projects/7ccfdd683cc748d98b42ca7a871efde2/users/fdec151fcdda46deb71f063ee2f1e39b/roles/e1c88eebe911432cab6a07fbeb84653b` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:43:02.813068 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8b4274-9c98-4654-b1f8-bd04e3fe06ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ccfdd683cc748d98b42ca7a871efde2, user_id=fdec151fcdda46deb71f063ee2f1e39b, role_id=e1c88eebe911432cab6a07fbeb84653b)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:43:02.815767 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8b4274-9c98-4654-b1f8-bd04e3fe06ac admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:43:02.920814 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1448/2903] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jun 29 11:43:02 2022] PUT /identity/v3/projects/7ccfdd683cc748d98b42ca7a871efde2/users/fdec151fcdda46deb71f063ee2f1e39b/roles/e1c88eebe911432cab6a07fbeb84653b => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 11:43:02.931288 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ba5b0c-8cbc-4e09-ab8b-5c95ae243790 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:43:02.931572 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ba5b0c-8cbc-4e09-ab8b-5c95ae243790 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:43:02.931871 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ba5b0c-8cbc-4e09-ab8b-5c95ae243790 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:43:02.958226 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-d9ba5b0c-8cbc-4e09-ab8b-5c95ae243790 None None] MFA Rules not processed for user `fdec151fcdda46deb71f063ee2f1e39b`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:43:02.959433 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-d9ba5b0c-8cbc-4e09-ab8b-5c95ae243790 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:43:03.022581 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1456/2904] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:43:02 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 11:43:03.033523 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8b4274-9c98-4654-b1f8-bd04e3fe06ac admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:43:03.035254 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-ea8b4274-9c98-4654-b1f8-bd04e3fe06ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:43:03.091223 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8b4274-9c98-4654-b1f8-bd04e3fe06ac admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:43:03.094872 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dee2fa1-de49-4f67-a940-9729c4083744 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff7931b75a924394b1af733febce0834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:43:03.096050 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dee2fa1-de49-4f67-a940-9729c4083744 service cinder] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:43:03.096333 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dee2fa1-de49-4f67-a940-9729c4083744 service cinder] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:43:03.096616 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dee2fa1-de49-4f67-a940-9729c4083744 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:43:03.104853 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dee2fa1-de49-4f67-a940-9729c4083744 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:43:03.106288 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dee2fa1-de49-4f67-a940-9729c4083744 service cinder] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:43:03.115871 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1449/2905] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:43:03 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:43:13.019542 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=45,keystone:INSERT=3 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:43:13.111590 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=51,keystone:INSERT=2 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:43:28.180723 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ba5b0c-8cbc-4e09-ab8b-5c95ae243790 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:43:28.242856 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ba5b0c-8cbc-4e09-ab8b-5c95ae243790 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:43:28.247348 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239df1c1-5399-476c-b9e2-6bc5100fac63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:43:28.248248 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239df1c1-5399-476c-b9e2-6bc5100fac63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:43:28.248534 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239df1c1-5399-476c-b9e2-6bc5100fac63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:43:28.248811 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239df1c1-5399-476c-b9e2-6bc5100fac63 admin admin] PATH_INFO: `/v3/users/fdec151fcdda46deb71f063ee2f1e39b` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:43:28.250000 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239df1c1-5399-476c-b9e2-6bc5100fac63 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fdec151fcdda46deb71f063ee2f1e39b)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:43:28.251949 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239df1c1-5399-476c-b9e2-6bc5100fac63 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:43:28.464732 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-239df1c1-5399-476c-b9e2-6bc5100fac63 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fdec151fcdda46deb71f063ee2f1e39b'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:43:28.465389 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-239df1c1-5399-476c-b9e2-6bc5100fac63 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fdec151fcdda46deb71f063ee2f1e39b'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:43:28.559403 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-239df1c1-5399-476c-b9e2-6bc5100fac63 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fdec151fcdda46deb71f063ee2f1e39b'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:43:28.567053 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-239df1c1-5399-476c-b9e2-6bc5100fac63 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fdec151fcdda46deb71f063ee2f1e39b'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:43:28.567702 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-239df1c1-5399-476c-b9e2-6bc5100fac63 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fdec151fcdda46deb71f063ee2f1e39b'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:43:28.585706 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1457/2906] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jun 29 11:43:28 2022] DELETE /identity/v3/users/fdec151fcdda46deb71f063ee2f1e39b => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:43:28.590260 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a07bc6c-a7e7-420a-bed5-1f718cc06493 None None] REQUEST_METHOD: `POST` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:43:28.590767 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a07bc6c-a7e7-420a-bed5-1f718cc06493 None None] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:43:28.590883 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a07bc6c-a7e7-420a-bed5-1f718cc06493 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:43:28.607611 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.auth.core [None req-3a07bc6c-a7e7-420a-bed5-1f718cc06493 None None] MFA Rules not processed for user `cdb4139035bd4d46a35dca750b630776`. Rule list: `[]` (Enabled: `True`). {{(pid=80165) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 11:43:28.608862 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-3a07bc6c-a7e7-420a-bed5-1f718cc06493 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:43:28.666341 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1450/2907] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jun 29 11:43:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 11:43:28.676001 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239df1c1-5399-476c-b9e2-6bc5100fac63 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:43:28.677477 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-239df1c1-5399-476c-b9e2-6bc5100fac63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:43:28.735179 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239df1c1-5399-476c-b9e2-6bc5100fac63 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:43:28.738714 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285865bf-5d93-4215-b76a-aba336cc56d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f4bb9f989e4024a5da28dced646b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:43:28.739612 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285865bf-5d93-4215-b76a-aba336cc56d3 service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:43:28.739921 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285865bf-5d93-4215-b76a-aba336cc56d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:43:28.740198 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285865bf-5d93-4215-b76a-aba336cc56d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:43:28.749484 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285865bf-5d93-4215-b76a-aba336cc56d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:43:28.750925 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285865bf-5d93-4215-b76a-aba336cc56d3 service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:43:28.760742 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1458/2908] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:43:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:43:29.210446 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a07bc6c-a7e7-420a-bed5-1f718cc06493 None None] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:43:29.211939 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-3a07bc6c-a7e7-420a-bed5-1f718cc06493 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:43:29.224922 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a07bc6c-a7e7-420a-bed5-1f718cc06493 None None] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:43:29.229496 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9309aa85-976c-4f04-a35b-7c4029f2b1c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:43:29.230502 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9309aa85-976c-4f04-a35b-7c4029f2b1c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:43:29.230866 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9309aa85-976c-4f04-a35b-7c4029f2b1c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:43:29.231167 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9309aa85-976c-4f04-a35b-7c4029f2b1c4 admin admin] PATH_INFO: `/v3/projects/7ccfdd683cc748d98b42ca7a871efde2` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:43:29.232344 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9309aa85-976c-4f04-a35b-7c4029f2b1c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ccfdd683cc748d98b42ca7a871efde2)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:43:29.234298 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9309aa85-976c-4f04-a35b-7c4029f2b1c4 admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:43:29.397365 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-9309aa85-976c-4f04-a35b-7c4029f2b1c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ccfdd683cc748d98b42ca7a871efde2', 'request_id': 'req-9309aa85-976c-4f04-a35b-7c4029f2b1c4'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:43:29.477214 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-9309aa85-976c-4f04-a35b-7c4029f2b1c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ccfdd683cc748d98b42ca7a871efde2', 'request_id': 'req-9309aa85-976c-4f04-a35b-7c4029f2b1c4'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:43:29.483762 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-9309aa85-976c-4f04-a35b-7c4029f2b1c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ccfdd683cc748d98b42ca7a871efde2', 'request_id': 'req-9309aa85-976c-4f04-a35b-7c4029f2b1c4'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:43:29.494773 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-9309aa85-976c-4f04-a35b-7c4029f2b1c4 admin admin] The token cache is being invalidate because project 7ccfdd683cc748d98b42ca7a871efde2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 11:43:29.500918 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-9309aa85-976c-4f04-a35b-7c4029f2b1c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:43:29.500918 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-9309aa85-976c-4f04-a35b-7c4029f2b1c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:43:29.500918 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1451/2909] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jun 29 11:43:29 2022] DELETE /identity/v3/projects/7ccfdd683cc748d98b42ca7a871efde2 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:43:38.756545 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:43:39.491852 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:43:55.063125 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285865bf-5d93-4215-b76a-aba336cc56d3 service neutron] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:43:55.064547 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-285865bf-5d93-4215-b76a-aba336cc56d3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:43:55.132363 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285865bf-5d93-4215-b76a-aba336cc56d3 service neutron] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:43:55.191907 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56daa43e-8943-49d4-a148-842165e99963 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a0505342b704d02977a5125957c2c9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:43:55.192637 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56daa43e-8943-49d4-a148-842165e99963 service placement] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:43:55.192920 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56daa43e-8943-49d4-a148-842165e99963 service placement] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:43:55.193305 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56daa43e-8943-49d4-a148-842165e99963 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:43:55.195127 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-56daa43e-8943-49d4-a148-842165e99963 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:43:55.206447 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56daa43e-8943-49d4-a148-842165e99963 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:43:55.208545 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56daa43e-8943-49d4-a148-842165e99963 service placement] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:43:55.218258 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1459/2910] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:43:55 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:44:05.213967 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=58 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:44:51.744607 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9309aa85-976c-4f04-a35b-7c4029f2b1c4 admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:44:51.746111 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-9309aa85-976c-4f04-a35b-7c4029f2b1c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:44:51.802105 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9309aa85-976c-4f04-a35b-7c4029f2b1c4 admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:44:51.805628 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d21f57b-73fa-45d1-a2ac-cf33edcd3215 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f4bb9f989e4024a5da28dced646b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:44:51.814074 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d21f57b-73fa-45d1-a2ac-cf33edcd3215 service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:44:51.814074 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d21f57b-73fa-45d1-a2ac-cf33edcd3215 service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:44:51.814074 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d21f57b-73fa-45d1-a2ac-cf33edcd3215 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:44:51.814074 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-2d21f57b-73fa-45d1-a2ac-cf33edcd3215 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:44:51.818145 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d21f57b-73fa-45d1-a2ac-cf33edcd3215 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:44:51.819656 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d21f57b-73fa-45d1-a2ac-cf33edcd3215 service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:44:51.828427 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1452/2911] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:44:51 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:44:53.290397 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56daa43e-8943-49d4-a148-842165e99963 service placement] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:44:53.300324 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56daa43e-8943-49d4-a148-842165e99963 service placement] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:44:53.303975 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1cbd4c-cb28-4039-bf3f-6b853bc51062 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f4bb9f989e4024a5da28dced646b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:44:53.304618 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1cbd4c-cb28-4039-bf3f-6b853bc51062 service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:44:53.304900 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1cbd4c-cb28-4039-bf3f-6b853bc51062 service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:44:53.305194 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1cbd4c-cb28-4039-bf3f-6b853bc51062 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:44:53.306996 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-1b1cbd4c-cb28-4039-bf3f-6b853bc51062 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:44:53.317171 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1cbd4c-cb28-4039-bf3f-6b853bc51062 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:44:53.318602 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1cbd4c-cb28-4039-bf3f-6b853bc51062 service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:44:53.328196 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1460/2912] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:44:53 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:44:53.403139 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d21f57b-73fa-45d1-a2ac-cf33edcd3215 service neutron] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:44:53.412681 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d21f57b-73fa-45d1-a2ac-cf33edcd3215 service neutron] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:44:53.416337 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809be4fe-42be-4e94-9b0a-584379ae196c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f4bb9f989e4024a5da28dced646b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:44:53.416972 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809be4fe-42be-4e94-9b0a-584379ae196c service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:44:53.417249 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809be4fe-42be-4e94-9b0a-584379ae196c service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:44:53.417527 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809be4fe-42be-4e94-9b0a-584379ae196c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:44:53.421293 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-809be4fe-42be-4e94-9b0a-584379ae196c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:44:53.489747 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809be4fe-42be-4e94-9b0a-584379ae196c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:44:53.491438 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809be4fe-42be-4e94-9b0a-584379ae196c service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:44:53.502078 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1453/2913] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:44:53 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:45:00.349149 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1cbd4c-cb28-4039-bf3f-6b853bc51062 service neutron] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:45:00.357927 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1cbd4c-cb28-4039-bf3f-6b853bc51062 service neutron] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:45:00.361577 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804be41d-93af-475c-ae37-68bd2c73e104 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f4bb9f989e4024a5da28dced646b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:45:00.362236 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804be41d-93af-475c-ae37-68bd2c73e104 service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:45:00.362521 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804be41d-93af-475c-ae37-68bd2c73e104 service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:45:00.362799 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804be41d-93af-475c-ae37-68bd2c73e104 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:45:00.364710 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-804be41d-93af-475c-ae37-68bd2c73e104 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:45:00.378665 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804be41d-93af-475c-ae37-68bd2c73e104 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:45:00.380145 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804be41d-93af-475c-ae37-68bd2c73e104 service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:45:00.391643 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1461/2914] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:45:00 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:45:03.497387 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=60 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:45:10.386131 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=12 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:45:55.789436 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809be4fe-42be-4e94-9b0a-584379ae196c service neutron] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:45:55.798090 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809be4fe-42be-4e94-9b0a-584379ae196c service neutron] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:45:55.802364 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521ee7fd-d78e-4e18-b6e2-13302b99aa29 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f4bb9f989e4024a5da28dced646b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:45:55.803001 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521ee7fd-d78e-4e18-b6e2-13302b99aa29 service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:45:55.803286 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521ee7fd-d78e-4e18-b6e2-13302b99aa29 service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:45:55.803560 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521ee7fd-d78e-4e18-b6e2-13302b99aa29 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:45:55.805365 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-521ee7fd-d78e-4e18-b6e2-13302b99aa29 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:45:55.814853 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521ee7fd-d78e-4e18-b6e2-13302b99aa29 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:45:55.816396 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521ee7fd-d78e-4e18-b6e2-13302b99aa29 service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:45:55.824586 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1454/2915] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:45:55 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:46:05.820848 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=6 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:46:39.911108 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804be41d-93af-475c-ae37-68bd2c73e104 service neutron] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:46:39.912841 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-804be41d-93af-475c-ae37-68bd2c73e104 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:46:39.923816 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804be41d-93af-475c-ae37-68bd2c73e104 service neutron] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:46:39.927375 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43239ce-dd62-4b9d-9d2d-3072793e8801 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f4bb9f989e4024a5da28dced646b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:46:39.928051 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43239ce-dd62-4b9d-9d2d-3072793e8801 service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:46:39.928332 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43239ce-dd62-4b9d-9d2d-3072793e8801 service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:46:39.928608 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43239ce-dd62-4b9d-9d2d-3072793e8801 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:46:39.930346 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-c43239ce-dd62-4b9d-9d2d-3072793e8801 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:46:39.988414 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43239ce-dd62-4b9d-9d2d-3072793e8801 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:46:39.990329 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43239ce-dd62-4b9d-9d2d-3072793e8801 service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:46:39.999537 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1462/2916] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:46:39 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:46:40.928144 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521ee7fd-d78e-4e18-b6e2-13302b99aa29 service neutron] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:46:40.929896 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-521ee7fd-d78e-4e18-b6e2-13302b99aa29 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:46:40.947363 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521ee7fd-d78e-4e18-b6e2-13302b99aa29 service neutron] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:46:40.953413 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef4e71f-c88f-4ec7-91e5-e81f6109936f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '796aeeff16df4739bddec1f6dee61bfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:46:40.954137 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef4e71f-c88f-4ec7-91e5-e81f6109936f service nova] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:46:40.954523 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef4e71f-c88f-4ec7-91e5-e81f6109936f service nova] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:46:40.954891 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef4e71f-c88f-4ec7-91e5-e81f6109936f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:46:40.956786 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-6ef4e71f-c88f-4ec7-91e5-e81f6109936f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:46:40.969742 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef4e71f-c88f-4ec7-91e5-e81f6109936f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:46:40.971368 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef4e71f-c88f-4ec7-91e5-e81f6109936f service nova] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:46:40.982944 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1455/2917] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:46:40 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:46:42.579677 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43239ce-dd62-4b9d-9d2d-3072793e8801 service neutron] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:46:42.588414 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43239ce-dd62-4b9d-9d2d-3072793e8801 service neutron] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:46:42.591999 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29f0937-dac0-4f2d-b17a-44fa00496cf8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f4bb9f989e4024a5da28dced646b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:46:42.592640 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29f0937-dac0-4f2d-b17a-44fa00496cf8 service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:46:42.592922 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29f0937-dac0-4f2d-b17a-44fa00496cf8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:46:42.593198 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29f0937-dac0-4f2d-b17a-44fa00496cf8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:46:42.595184 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-d29f0937-dac0-4f2d-b17a-44fa00496cf8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:46:42.610114 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29f0937-dac0-4f2d-b17a-44fa00496cf8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:46:42.611018 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29f0937-dac0-4f2d-b17a-44fa00496cf8 service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:46:42.623623 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1463/2918] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:46:42 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:46:46.472189 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef4e71f-c88f-4ec7-91e5-e81f6109936f service nova] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:46:46.481504 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef4e71f-c88f-4ec7-91e5-e81f6109936f service nova] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:46:46.485065 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dcfe1c-7b66-49a0-9d22-2a2b727ff27c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5f4bb9f989e4024a5da28dced646b92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:46:46.485695 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dcfe1c-7b66-49a0-9d22-2a2b727ff27c service neutron] REQUEST_METHOD: `GET` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:46:46.485979 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dcfe1c-7b66-49a0-9d22-2a2b727ff27c service neutron] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:46:46.486255 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dcfe1c-7b66-49a0-9d22-2a2b727ff27c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:46:46.488083 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-91dcfe1c-7b66-49a0-9d22-2a2b727ff27c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:46:46.550039 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91dcfe1c-7b66-49a0-9d22-2a2b727ff27c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:46:46.551470 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91dcfe1c-7b66-49a0-9d22-2a2b727ff27c service neutron] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:46:46.561021 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1456/2919] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:46:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:46:52.618084 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=34 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:46:53.588305 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29f0937-dac0-4f2d-b17a-44fa00496cf8 service neutron] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:46:53.590733 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-d29f0937-dac0-4f2d-b17a-44fa00496cf8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:46:53.609332 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29f0937-dac0-4f2d-b17a-44fa00496cf8 service neutron] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:46:53.614288 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b97db6c-f4a7-4007-b373-1df59fec49ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:46:53.615199 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b97db6c-f4a7-4007-b373-1df59fec49ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:46:53.615487 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b97db6c-f4a7-4007-b373-1df59fec49ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:46:53.615769 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b97db6c-f4a7-4007-b373-1df59fec49ce admin admin] PATH_INFO: `/v3/users/f9db0a0b5bcf4e6d8ee2593b21c415fb` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:46:53.617276 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b97db6c-f4a7-4007-b373-1df59fec49ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=f9db0a0b5bcf4e6d8ee2593b21c415fb)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:46:53.619233 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b97db6c-f4a7-4007-b373-1df59fec49ce admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:46:53.845845 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-5b97db6c-f4a7-4007-b373-1df59fec49ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f9db0a0b5bcf4e6d8ee2593b21c415fb'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:46:53.846457 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-5b97db6c-f4a7-4007-b373-1df59fec49ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f9db0a0b5bcf4e6d8ee2593b21c415fb'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:46:53.930223 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-5b97db6c-f4a7-4007-b373-1df59fec49ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f9db0a0b5bcf4e6d8ee2593b21c415fb'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:46:53.937614 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-5b97db6c-f4a7-4007-b373-1df59fec49ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f9db0a0b5bcf4e6d8ee2593b21c415fb'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:46:53.938255 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-5b97db6c-f4a7-4007-b373-1df59fec49ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f9db0a0b5bcf4e6d8ee2593b21c415fb'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:46:53.955929 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1464/2920] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jun 29 11:46:53 2022] DELETE /identity/v3/users/f9db0a0b5bcf4e6d8ee2593b21c415fb => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 11:46:54.194213 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dcfe1c-7b66-49a0-9d22-2a2b727ff27c service neutron] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:46:54.204288 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dcfe1c-7b66-49a0-9d22-2a2b727ff27c service neutron] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:46:54.265744 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6f6f05-6fd1-4a3a-afd7-84a519235b6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:46:54.266599 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6f6f05-6fd1-4a3a-afd7-84a519235b6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:46:54.266882 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6f6f05-6fd1-4a3a-afd7-84a519235b6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:46:54.267173 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6f6f05-6fd1-4a3a-afd7-84a519235b6c admin admin] PATH_INFO: `/v3/projects/40d5bbf32cb54de2a37895f53bcb9d4a` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:46:54.268394 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6f6f05-6fd1-4a3a-afd7-84a519235b6c admin admin] RBAC: Authorizing `identity:delete_project(project_id=40d5bbf32cb54de2a37895f53bcb9d4a)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:46:54.270365 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6f6f05-6fd1-4a3a-afd7-84a519235b6c admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:46:54.382455 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-1b6f6f05-6fd1-4a3a-afd7-84a519235b6c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '40d5bbf32cb54de2a37895f53bcb9d4a', 'request_id': 'req-1b6f6f05-6fd1-4a3a-afd7-84a519235b6c'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:46:54.484665 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-1b6f6f05-6fd1-4a3a-afd7-84a519235b6c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '40d5bbf32cb54de2a37895f53bcb9d4a', 'request_id': 'req-1b6f6f05-6fd1-4a3a-afd7-84a519235b6c'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:46:54.492160 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-1b6f6f05-6fd1-4a3a-afd7-84a519235b6c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '40d5bbf32cb54de2a37895f53bcb9d4a', 'request_id': 'req-1b6f6f05-6fd1-4a3a-afd7-84a519235b6c'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:46:54.502230 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-1b6f6f05-6fd1-4a3a-afd7-84a519235b6c admin admin] The token cache is being invalidate because project 40d5bbf32cb54de2a37895f53bcb9d4a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 11:46:54.502518 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-1b6f6f05-6fd1-4a3a-afd7-84a519235b6c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:46:54.503141 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-1b6f6f05-6fd1-4a3a-afd7-84a519235b6c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:46:54.504724 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1457/2921] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jun 29 11:46:54 2022] DELETE /identity/v3/projects/40d5bbf32cb54de2a37895f53bcb9d4a => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:46:54.509357 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b97db6c-f4a7-4007-b373-1df59fec49ce admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:46:54.511348 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-5b97db6c-f4a7-4007-b373-1df59fec49ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:46:54.523460 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b97db6c-f4a7-4007-b373-1df59fec49ce admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:46:54.577766 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e917f075-7031-4a8f-88d1-496d66d7cf95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:46:54.578637 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e917f075-7031-4a8f-88d1-496d66d7cf95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:46:54.578955 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e917f075-7031-4a8f-88d1-496d66d7cf95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:46:54.579232 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e917f075-7031-4a8f-88d1-496d66d7cf95 admin admin] PATH_INFO: `/v3/users/d28eb9f00e1044888bbe8cc953247068` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:46:54.580449 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e917f075-7031-4a8f-88d1-496d66d7cf95 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d28eb9f00e1044888bbe8cc953247068)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:46:54.582399 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e917f075-7031-4a8f-88d1-496d66d7cf95 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:46:54.766504 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-e917f075-7031-4a8f-88d1-496d66d7cf95 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd28eb9f00e1044888bbe8cc953247068'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:46:54.767149 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-e917f075-7031-4a8f-88d1-496d66d7cf95 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd28eb9f00e1044888bbe8cc953247068'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:46:54.871486 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-e917f075-7031-4a8f-88d1-496d66d7cf95 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd28eb9f00e1044888bbe8cc953247068'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:46:54.878879 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-e917f075-7031-4a8f-88d1-496d66d7cf95 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd28eb9f00e1044888bbe8cc953247068'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:46:54.879536 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-e917f075-7031-4a8f-88d1-496d66d7cf95 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd28eb9f00e1044888bbe8cc953247068'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:46:54.887440 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:46:54.896731 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1465/2922] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jun 29 11:46:54 2022] DELETE /identity/v3/users/d28eb9f00e1044888bbe8cc953247068 => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:46:55.234232 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6f6f05-6fd1-4a3a-afd7-84a519235b6c admin admin] Authenticating user token {{(pid=80165) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:46:55.235702 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.fernet_utils [None req-1b6f6f05-6fd1-4a3a-afd7-84a519235b6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80165) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:46:55.249187 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6f6f05-6fd1-4a3a-afd7-84a519235b6c admin admin] Validating token access rules against request {{(pid=80165) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:46:55.258044 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:46:55.309115 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa94472-f659-451e-a084-9d4b545a8a9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdb4139035bd4d46a35dca750b630776', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37897f6664b94d219c11b87aa269041a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80165) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:46:55.310004 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa94472-f659-451e-a084-9d4b545a8a9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:46:55.310285 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa94472-f659-451e-a084-9d4b545a8a9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:46:55.310559 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa94472-f659-451e-a084-9d4b545a8a9f admin admin] PATH_INFO: `/v3/projects/7ec1045253fa42be8148bdeb29a8a1dc` {{(pid=80165) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:46:55.311752 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa94472-f659-451e-a084-9d4b545a8a9f admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ec1045253fa42be8148bdeb29a8a1dc)` {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:46:55.313700 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa94472-f659-451e-a084-9d4b545a8a9f admin admin] RBAC: Authorization granted {{(pid=80165) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:46:55.449784 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-cfa94472-f659-451e-a084-9d4b545a8a9f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ec1045253fa42be8148bdeb29a8a1dc', 'request_id': 'req-cfa94472-f659-451e-a084-9d4b545a8a9f'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:46:55.556997 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-cfa94472-f659-451e-a084-9d4b545a8a9f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ec1045253fa42be8148bdeb29a8a1dc', 'request_id': 'req-cfa94472-f659-451e-a084-9d4b545a8a9f'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:46:55.563447 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-cfa94472-f659-451e-a084-9d4b545a8a9f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ec1045253fa42be8148bdeb29a8a1dc', 'request_id': 'req-cfa94472-f659-451e-a084-9d4b545a8a9f'} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:46:55.574186 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-cfa94472-f659-451e-a084-9d4b545a8a9f admin admin] The token cache is being invalidate because project 7ec1045253fa42be8148bdeb29a8a1dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80165) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 11:46:55.574474 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-cfa94472-f659-451e-a084-9d4b545a8a9f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:46:55.575103 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG keystone.notifications [None req-cfa94472-f659-451e-a084-9d4b545a8a9f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80165) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 11:46:55.576676 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: [pid: 80165|app: 0|req: 1458/2923] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jun 29 11:46:55 2022] DELETE /identity/v3/projects/7ec1045253fa42be8148bdeb29a8a1dc => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 11:47:05.238714 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=3,keystone:DELETE=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:47:05.637984 dv-cinder-847335-2-iscsi devstack@keystone.service[80165]: DEBUG dbcounter [-] [80165] Writing DB stats keystone:SELECT=46,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80165) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 11:49:01.088448 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e917f075-7031-4a8f-88d1-496d66d7cf95 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 11:49:01.089885 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-e917f075-7031-4a8f-88d1-496d66d7cf95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:49:01.227410 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e917f075-7031-4a8f-88d1-496d66d7cf95 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 11:49:01.284816 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f3d344-864f-49e1-afef-9bd46bf2a5c6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a0505342b704d02977a5125957c2c9e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebe5dffa4bb8440286687daefb6c98f4', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 11:49:01.285469 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f3d344-864f-49e1-afef-9bd46bf2a5c6 service placement] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 11:49:01.285757 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f3d344-864f-49e1-afef-9bd46bf2a5c6 service placement] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 11:49:01.286039 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f3d344-864f-49e1-afef-9bd46bf2a5c6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 11:49:01.287868 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-27f3d344-864f-49e1-afef-9bd46bf2a5c6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 11:49:01.298918 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f3d344-864f-49e1-afef-9bd46bf2a5c6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 11:49:01.300542 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f3d344-864f-49e1-afef-9bd46bf2a5c6 service placement] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 11:49:01.310261 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1466/2924] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jun 29 11:49:01 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 11:49:11.306016 dv-cinder-847335-2-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=89 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}